New Zealand was a multi-denominational society, but he said he had been "pretty angry" at the removal of reference to the Queen and Jesus Christ.
The National MP said he had done what he could - he told the Speaker his opinion, and joined the National Party caucus collective view it should not change - but respected it was the Speaker's decision.
"I just don't think in terms of things that need to be changed and priorities of Parliament that was necessarily one of them at this stage in the process."
Napier MP Stuart Nash however had "no issues" with the changes, and said the matter was up to the Speaker.
"I've got a whole lot more important issues that I've got to deal with than worrying about the parliamentary prayer, but I certainly don't have any concerns," the Labour MP said.
"We are supposed to be a Parliament which is inclusive of all religions and ethnicities and if this is one small thing we can do to promote diversity then why not."
Others stressed the need to ensure the prayer reflected New Zealand as a secular society.
As a practicing Anglican, Ikaroa-Rawhiti MP Meka Whaitiri said the tradition of the prayer was important to her but understood many New Zealanders did not identify as Christian.
"It is one of those tricky issues. We're going to all have different views and that's not unhealthy, that means democracy is working and people are very passionate about it.
"I'm happy if [the Speaker] leaves it in, I'm happy if he leaves it out, but for me it's important that I keep my faith."
Different voices needed to be heard, which was why a process of open engagement before any such decisions were made was important, she said: "It is the Speaker's right as it is his Parliament, but it does belong to all New Zealanders."
National's Wairarapa MP Alastair Scott said he was ok with the change, but glad the speaker had reinstated the reference to the Queen.
"[I was] disappointed Jesus Christ was left out but I understand and accept the change given we are a multi-cultural society made up of multiple religions and beliefs."
Labour list MP for Wairarapa Kieran McAnulty said he thought having a reference to God, but not specifically Jesus Christ made the prayer more inclusive.
Although he respected the change was an issue to some, the self-described "lapsed" Catholic said while most New Zealanders were Christian, there were many who believed in other religions, or did not follow any.
"Parliament should reflect the country, it's called the House of Representatives and as a result I think the changes do reflect New Zealand today without taking away our traditions."
Wairarapa list MP and New Zealand First deputy leader and Ron Mark could not be reached yesterday.
Mr Mark has previously told NZME he wanted to keep references to the Queen.
According to the 2013 census 49 per cent of New Zealanders who answered the question on religion identified themselves as Christian.
The proportion of Hawke's Bay residents who affiliated with a religion was 56 per cent. Anglican was the highest reported religion in Hawke's Bay with 16.5 per cent of the population, followed by Catholic at 11.4 per cent.
The prayer acknowledges the need for guidance from "Almighty God", and to conduct affairs "to the glory of Thy holy name, the maintenance of true religion and justice, the honour of the Queen, and the public welfare, peace, and tranquillity of New Zealand, through Jesus Christ our Lord."
Changing the prayer was debated, then dropped in 2014. In 2007, 63 per cent of MPs voted to keep the prayer unchanged.