The fishermen then alerted the authorities, who dispatched a helicopter to the area, known as Manta Point.
The first to be rescued was 27-year-old Saori Furukawa, who had been separated from the others.
The helicopter was unable to reach the others so two rescuers leaped into the water to take them supplies of food. The five survivors were rushed to hospital in Bali, press images showing them strapped into stretchers with their heads in braces. But officials said that while they have suffered minor abrasions, none of the divers was in serious condition.
The dive boat's skipper said that after the divers submerged he had followed them for around 20 minutes but lost sight of them when a sudden downpour churned the water. He said he then took the boat to an agreed meeting point a few hundred metres away, but the divers failed to surface.
After searching for them for around an hour, he reported the incident to authorities, he said.
But Hamzar said he had received information that the skipper had run out of fuel while waiting for the divers, and had to leave the area to refuel before returning to the meeting point.
The diving expedition was run by Yellow Scuba, an operator set up by Shoko Takahashi, one of the instructors with the group, and her husband, according to a Japanese consular official.
Furukawa is also a locally-based instructor while the others are tourists. The other four divers rescued were Emi Yamamoto 33, Nahomi Tomita, 29, Aya Morizono, 60, and Atsumi Yoshidome, 29. Three of the women are nurses at Kobe University Hospital. Takahashi is believed to be among the missing divers, along with Ritsuko Miyata, 59.