The man, believed to be a Chinese national, fell 10m into the harbour and did not resurface about 9.20am today.
Police and Coastguard searched for the man without success.
A navy ship happened to be in port at the time and navy divers have joined the search. Senior Sergeant Karl Konlechner of the Bay of Plenty District Command Centre said it appeared to be "a tragic accident".
"The search has yet to locate the man but divers are now looking for his body. The police are very grateful for the prompt response from the NZ Navy personnel who were in the port at the time and quickly joined the search."