While patrolling streets he often came across Muslims, who passing through, had pulled over at the train station or on the side of the road to pray. (Muslims pray five times a day.)
"There's a lot of Muslim travellers who stop here," he said.
Mr Abu started talking to them and even invited a few to use his home for prayer.
"It was kind of bleak for them to [just] hang around."
There was no mosque between Taupo and Palmerston North or Wanganui.
Soon he was receiving calls from Muslims wanting to use his home to pray.
"My phone kept ringing. For the first week it was fine [but] we needed to think about a long- term solution."
Being on the main state highway made Taihape the perfect location for a mosque for transient Muslims.
An old shop across the road from Mr Abu's work on Tui St had been vacant for a while and was on the market. Mr Abu contacted the Federation of Islamic Associations of NZ (FIANZ) and sought approval before gathering donations.
He raised $80,000 from all over the world which helped pay for the building and set up.
The mosque opened on Saturday with a lunch and open day.
His mother and mother-in-law travelled from Singapore to be at the opening.
Ad-Deen claims to be the highest mosque in New Zealand at 440m above sea level.
The thick, soft carpet inside the mosque comes from Saudi Arabia and is worth as much, if not more, than the building. Mr Abu said the mosque would also be used as knowledge centre where people could come to learn about Islam.
The project had been well supported by the Taihape community, he said.
There had been talk within the Muslim community about more rural mosques popping up. "It's sort of created a precedent."
And just as it opens, Mr Abu already has plans to extend the mosque, creating more space for prayer and separation, mainly for women.
FIANZ has already put $10,000 towards the expansion.
And Mr Abu has no plans to leave Taihape.
"We love this place. It's a quiet town, people know each other by name, which is good. It's more rural and community-based here. When I go to a job, I make sure I get the job right from beginning to end because I'm going to have to see those people again."