Mr Kemp, who moved to the United States in 2010, then began researching one of the most valuable substances in nature, before publishing his book Floating Gold: A Natural (and Unnatural) History of Ambergris.
Ambergris was the result of "a series of highly unlikely oddities". It was produced by just a few hundred sperm whales and released at sea.
"The odds of finding it are minute and so its value is high. Something like it can be produced in a laboratory, but it's ... more like an approximation."
New Zealand is known for high-quality ambergris, probably because it is an isolated land mass surrounding by deep waters in which sperm whales were relatively common.
In particular, Stewart Island was known for quality ambergris, as was the west coast of the North Island, in areas around Dargaville and along Ninety Mile Beach.
People were collecting and trading ambergris full-time like any other commodity, but "they're very tight-lipped folks".
And, in some cases, irate ones: "Especially the friendly and loving people of Stewart Island, who sent me hate-mail and threats of legal action after my book was published," Mr Kemp said.
Good quality ambergris could sell for thousands of dollars a kilogram and, while he had never found any, "somewhere down the line, someone is getting rich".
Mr Kemp said he enjoyed writing the book: "I'll always be glad I did it. I just might not be welcome on Stewart Island any time soon."
Natural history teacher and author Lloyd Esler, of Invercargill, said he had found pieces of ambergris over the years and advised fossickers "they have to smell a lot of dog droppings before they find their first piece of ambergris".
He said the popularity of ambergris-hunting in the South had increased thanks to publicity about its value and he had heard of people training dogs to sniff it out.
Hotspots in the south were Oreti Beach, near Invercargill, and Mason Bay on Stewart Island - "but they are also the two beaches frequently scoured by people who know what the stuff is".
"You have to get your eye in for it, as there is an awful lot of other stuff on the beach that you could step over. There was report of a big find on Stewart Island and there were footprints over and around it, so people did not know what it was."
Mr Esler had just finished writing a book Whaling and Sealing in Southern New Zealand, which noted whalers would slit open sperm whales' innards as part of processing.
The total ambergris recovered by all whaling vessels from 1841 to 1914 was said to be 1990kg, including 446kg collected from whales by the Splendid on a voyage in 1882. It was sold for GBP25,000.
Other ambergris finds reported in the South include:
• In 1928, three Southland men found an 85kg lump of ambergris on a beach near Otara, Southland, earning them GBP8000 from a French perfume company. After expenses were deducted, themen had enough to buy their own farms.
• In 1945, a 27kg piece of ambergris was found on Stewart Island.
• In 2006, 10-year-old Dunedin boy found an 860g lump of ambergris at Purakaunui, and another 370g the next day, potentially earning him $10,000.