Australian veteran Jack Tredrea, 96, proudly unveiled the plaque and remembered his mates.
"There are a lot of fellows looking down on us from upstairs with big smiles too," he told the ABC.
Families of some Kiwi veterans, who are all now dead, attended the ceremony, including Judi Millar, daughter of Sergeant Frank Wigzell who trained headhunter tribesmen in Japanese-held Borneo into a feared resistance fighter group.
She earlier said the men should also be officially recognised in New Zealand, perhaps with a similar memorial at Pukeahu National War Memorial Park in Wellington or through a posthumous medal.
"My father campaigned the Government right up until the day he died only to be turned down repeatedly," said Millar.
"The Australian men were highly decorated years later but the New Zealand men were denied everything. They were given a Pacific Star and that was supposed to be sufficient. What a disgusting thing to say to someone."
Z Special Unit, attached to the Australian Army and part of Special Operations Australia, was devoted to special operations including intelligence-gathering and guerrilla warfare.
It was considered the forerunner to the modern-day SAS (Special Air Service) and Commando units in New Zealand and Australian military.
In 2002, the New Zealand SAS commissioned its own memorial to World War II heroes at its Papakura Military Camp barracks in Auckland.
The families are allowed inside the top-secret army base on Anzac Day to lay wreaths and pay their respects.
But Millar believes there should be a more public memorial accessible to all New Zealanders.
"These guys put their lives on the line and when they came home, nobody wanted to know them. To honour them now is the least we can do, the humane thing to do."
A spokesman for the Ministry for Culture and Heritage said it is "happy to talk with groups who want to discuss such requests".
"Generally it is not policy to recognise individuals or groups of individuals with additional memorials on the principle of uniformity and equality of treatment in death for those who have died on war service," he said.
"The principle also applies to requests for special memorials to honour particular branches of the Armed Services such as Bomber Command."