"The doors got shut and he got locked in and then he was driven away," the wife said.
"They drove him into a property where they locked the gate. They took him inside and they sat [him] down, they wouldn't let him leave. The house was really little. People seemed to come and go within the room."
The Kiwi "started to get quite fretful" because he had no idea what the strangers were capable of.
"He tried to do what he was told because he was obviously quite scared by that stage," his wife said.
"He had no idea what was coming. He didn't think he was going to get out of it alive. He thought it was going to be over. They didn't say anything like that, but there was just too much going on."
Inside the house another Balinese man was playing cards with a woman.
The local man forced the New Zealander to play a round before telling him he had to "up the stakes" by betting money.
"They kept saying 'Oh you're winning, you're winning. You'll just have to keep playing for a little bit longer'."
When the tourist ran out of cash the scammers drove him to a shop and forced him to swipe his card and enter his pin, his wife said.
"He was quite frightened then because there was someone at the door so he knew he couldn't escape.
"He just tried to do everything to get out of it at this stage because he was really terrified. He swiped his card and they just took out every last bit of money."
The money they took from his account along with the cash totalled more than NZD $2000, his wife said.
After the four-and-a-half hour long ordeal the scammers finally put the man in a taxi and sent him back to his hotel. Before the car drove away they gave him a cellphone.
"They said 'We'll be in touch'."
When the man arrived back at his hotel, he handed the phone into reception and told them and his family what had happened.
They contacted a relative back home, who called the New Zealand Embassy in Jakarta.
The Embassy got the man to file a report with the local police and advised the family to return to New Zealand immediately, the wife said.
"It was like, 'we're not safe here'."
They cut their holiday a week short, forking out $2000 to change their flights because their insurance refused to pay despite them having comprehensive cover.
The wife said the experience had been "really terrifying" and wanted to warn other Kiwi travellers of the scam.
A Ministry of Foreign Affairs and Trade (Mfat) spokesman confirmed the family of a New Zealander contacted the New Zealand Embassy in Jakarta for consular assistance.
Travellers should check the Mfat website Safe Travel if they were concerned about being scammed while overseas and report any scams they are the victim of to the local police, the spokesman said.
The Safe Travel website warns of scams similar to the one that the family described.
"Many countries on the tourist beat have their own special scams that target the overseas traveller," it states.
"There are familiar tales of travellers... meeting someone on the street who wants to practice their English and then becoming an unwitting player in a 'high stakes' poker game...
"Things turn serious when you are unable to pay. You may be forced to the nearest ATM to settle your debt."
In 2011 an Australian couple who were holidaying in Bali were conned in similar circumstances.
The West Australian newspaper reported the scam cost Delys and Norm Langford, of Perth, AUD $18,000.
