One of the passengers Eduarda Carvalho, from Portugal, told of the panic on board after the explosion.
"About five minutes after we left we saw an explosion and we saw people with injuries ... it is not fire, only explosion and smoke," Ms Carvalho said.
She said it was scary and people panicked.
"The message is simply, Bali is not a good spot for tourists. I never will recommend Bali as a tourist spot," Ms Carvalho said, who added she uninjured and had helped others who were hurt.
The injured were ferried to clinics in Padangbai and then to Sanglah Hospital in Denpasar.
Initial reports suggested that a fuel tank may have exploded on board the boat.
The Karangasem police chief, Sugeng Sudarso told News Corp Australia the victims were evacuated to a clinic in nearby Manggis Village.
"There are many victims injured, mostly foreigners," Sudarso said.
"The boat departed from Padangbai at 9.35am. Five minute after they depart, around 200 metre from the harbour, there was smoke seen from the boat. Then there was an explosion, "he said.
Sudarso said the body had been taken to the Sanglah Hospital morgue in Denpasar. He said the victim was a woman but could not confirm reports that she was German.
He said police were still investigating the cause of the explosion.
"The explosion come from the fuel tank and the engine. We are still investigating the exact cause of the explosion," Sudarso said.
Putu Swardiana, from the Padangbai health agency, said: "The incident caused 20 foreigners to be injured and one has died."
The boat was brought back to the harbour where forensics police late today boarded the vessel to conduct inquiries.
It had been heading to Gili Air, Gili Trawangan and Teluk Kode, islands especially popular with divers.
"From 35 passengers, there are 17 passengers injured and two died."
Manifests, seen by News Corp Australia, showed that the passengers were Europeans - 13 Brits, four French, four Italians, four Austrians and two each from Spain, Portugal, Germany, Ireland and Holland.
A Ministry of Foreign Affairs spokesperson said no New Zealanders were involved and they have no further comment.