Lying 860km east of Christchurch, the Chathams are also home to many birds and plants that can only be found there. The distance from the mainland also provides some of the best fishing in the country.
Our tour party of 10 loopies - the affectionate term locals give to tourists - were taken out on the Hotel Chathams charter boat, in the safe hands of skipper Matt and deckhand Dylan. Both lads are in their 20s and work as commercial fisherman. Despite the depth finder being on the blink, they had us anchored over a spot that proved popular with the fish.
Within minutes of hurling our rope handlines over the side, we were pulling up hapuku - some weighing in at 15kg. Forget the flashy Auckland boat rods, the Chatham Island line - soon nicknamed the Chatham Shimano by our group - consisted of a few sinkers and a couple of huge hooks tied to strong braided rope. We caught our self-imposed quota in 30 minutes, filling several bins with 14 hapuku and 40-odd blue cod. The island is known for its kaimoana which frequently features on the Hotel Chathams menu, and many of the locals are employed in the crayfish, blue cod and paua industries.
As well as being renowned for its seafood, the island is also home to some rich history.
Hapupu, north-east of the main island, is where visitors will find the rakau momori, or Moriori dendroglyphs. Carved, gouged and bruised into live kopi trees - a close relative of the karaka - they are thought to represent ancestors and are hundreds of years old, pre-dating the arrival of Europeans and Maori. Standing in the grove is akin to visiting a cathedral - a feeling of awe and peace pervades the area.
But sadly, these national treasures are being lost. Many of the trees were felled in the 19th century to make way for farming and others have fallen to the rugged climate and stock grazing.
In the 1940s, anthropologist Christina Jefferson recorded 1145 of the carvings. The most recent survey in 1998 found just 185.
Some of the carvings have been removed and are housed in museums on the mainland, others have been taken away for garden ornaments.
The Moriori Kopinga marae is situated in the middle of the island with sweeping views of Lake Huro.
Hokotehi Moriori Trust member Shirley King - known as Nana Shirl - proudly showed me around the five-sided meeting house. She says the art of tree carving is being revived and this is evident in the unique carvings that adorn the posts at Kopinga.
"It is something we want to start again - like an apprenticeship but once again it comes down to the tyranny of distance and the tyranny of expense."
Many of her people now live away from the islands and some of them have the crucial skills and knowledge needed.
Nana Shirl says the Moriori are a peace-loving people and traditionally settled their disputes with hand to hand combat, the winner being the first to draw blood from his opponent. It is the oldest peace covenant still in existence.
When Taranaki Maori arrived in 1835, the Moriori responded with traditional Chatham Island hospitality. The visitors did not respond in kind and according to historian Michael King, about 300 Moriori were killed and those who survived were enslaved.
Europeans had found the Chathams some 40-odd years previously and colonial history has also left its mark on the land. Kaingaroa, the second township on the Chathams, is the site where the first Europeans landed. Lieutenant William Broughton sailed into the harbour in 1791 but after his men failed to find water and trade with the Moriori, a struggle resulted and a Moriori man was shot. A monument to the dead man, identified as Tamakaroro, stands in the town.
More Europeans followed to take advantage of sealing and whaling opportunities. A tripot can still be seen at Mission Bay where German missionaries settled and built stone houses.
These have not survived but another stone cottage is still standing on the other side of the island at Maunganui. Built by Johann Gottfried Engst and William Baucke in the early 1870s, this impressive landmark is still used.
We were fortunate to be shown around the Chathams by professional tour guide Neil Rawlins who frequently stopped the bus to tell us a tale about a local spot or identify the island's unique birdlife and plants.
One of the stories Neil told was of Blind Jim who was born into slavery and arrived in the Chathams in the 19th century after the whaling boat he was working on sank. He decided to stay and made his home among the Maori on the shore. A local landowner decided to help Jim out and employed the island doctor to operate on Jim's cataracts. The doctor - known as a drinker - was unsuccessful but at least Jim lived. Many of the doctor's other patients are reported to have not been as fortunate.
A stretch of Te Whanga, the island's lagoon, has been named after Blind Jim and today tourists and locals stop to hunt for 40 million year-old fossilised sharks' teeth preserved by limestone.
The Chathams is home to unique wildlife, plants and insects. It is also packed full of history. For New Zealanders who want to get off the beaten track and visit a part of the country's living heritage, it is a must-see.
Highlights of a Chatham Islands tour
Fishing and diving: The Chathams have some of the best fishing in the country. Hapuku and blue cod are in abundance, paua can be found on the rocks and the crayfish is a must.
Rakau momori: These tree carvings are internationally significant and are slowly being lost. It is a privilege to see them.
Kopinga Marae: A unique meeting house that celebrates the Moriori people - their heritage and their future.
Birdlife: The Chathams are home to a host of native birds including the taiko, Chatham Island black robin and Chatham Island oystercatcher, known as the torea.
The Chathams museum based in Waitangi is small but chock-full of interesting items. It includes a lot of information on Ringatu leader Te Kooti who was imprisoned on the island before leading an uprising and commandeering a supply ship to get back to the mainland.
Manukau Pt: Home of the statue to the last full-blooded Moriori, Tommy Solomon, who died in 1933. But the idea that the Moriori people ended with Tommy's passing is untrue, with many islanders identifying with the ethnic group.
Stone cottage: Built out of stone and mortar using burned pipi shells, this iconic house is still used today and is a testament to the skills of the men who built it in the 1860s.
Blind Jim's: On the shores of Te Whanga Lagoon can be found fossilised sharks' teeth estimated to be 40 million years old.
* Edward Gay travelled as a guest of Pukekohe Travel (09 237 0013), flying with Air Chathams and staying at Hotel Chatham.