Mr Glassford's wife, Karen Glassford, nee Sinnamon, is also from a proud Otago pioneering family.
"Unlike his lot, some of us trudged off the ship and kept on walking till we found a place to sit, but they were comfortable."
The family fortune had been greatly diminished by the time the Glassfords got to New Zealand.
The family farm has the unusual distinction of having a cemetery right smack in the middle, the last remnant of the frontier gold mining town of Drybread. It's full of Gordon Glassfords.
How the double-barrel name came about is as richly complex as the family's riches-to-rags story. Mr Glassford's ancestor was John Glassford (1715-1783), a Scottish merchant of Glasgow on a large scale with plantations in Virginia and Maryland.
He married three times and one of his estates around Glasgow was Dougalston, from which he took his title as laird.
Like many, he was involved in the slave trade that laid the foundation of Glasgow's commercial success through the tobacco trade, and interestingly, was all for the American Revolution — the War of Independence — even refusing to sell his ships to the English.
However, by 1783 his fortunes and health were in a poor state. When he died, his eldest son, Henry, inherited the Dougalston estate. He never married or had children, and seeing the writing on the wall, drew up a tailzie, an entail on the estate to keep it in a trust for his younger half-brother, James.
James also had no children. He died in 1845, and in accordance with the tailzie when there were no male heirs, the eldest male child of the eldest sister (Jean) would inherit.
She had married a Gordon (there's a connection with Gordon's gin), and her son was James Gordon.
A further condition of the tailzie was that the surname Glassford should be adopted by any heir. James Gordon became James Gordon Glassford. His brother, Henry, succeeded him, and then Henry's son, James, who inherited the title in 1860.
James Glassford Gordon Glassford, known in the family genealogy as JGGG, came to New Zealand with his brother, Harry, in the 1860s.
They operated as the Glassford Bros from 1864 to about 1872, purchasing 80,000 acres in the Lauder Survey District and taking over the lease of what was known as Run 223.
By then, JGGG was married to Margaret Thomson Bain and had two sons, Henry and William. Henry (1862-1907) is Mr Glassford's great-great-grandfather and is buried at the Drybread cemetery with his wife, Louisa (nee Waldron). She was a barmaid — and a Catholic.
Henry's mother, Margaret, did not approve of the relationship, and sold Run 223 from under him, meaning he had to start from scratch with the few acres left. And so the Gordon Glassford and Dougalston connection has come down the line to Tony Glassford through his grandfather, Oswald (1893-1983), and his father, Owen.
To Mr Glassford, the unusual origin of his legal surname of Gordon Glassford is part of who he is.
"It's important because it's the story of why my family is here today. Everyone is shaped by their family history even if they don't know what it is. I'm just lucky I have such a clear picture — literally — of the past and my family, warts and all, to connect with."
The name will end up on his headstone in the Drybread cemetery, although at 61, he hopes that will not be for a long while yet, as does his wife, Karen (54).
"Yes, but I would have the main title as Karen Glassford, and in the small print have my maiden name and full legal married name."
Their three children will be free to decide if they will pass the name on.
"All said they wouldn't like to make a decision that cut their children off from their history and would want each of them to choose for themselves, just as we want for our children," Mrs Glassford said.
"It would be nice if they want to carry it on as it's a pretty unique slice of history but neither of us are particularly worried, as after all, the name only survives today because of deliberate change," Mr Glassford said.
The Gordon Glassford name lives on in other branches of the family, but the Glassfords of Dougalston are the only remaining farming family of those original settler brothers.
The main issue going forward is one of identity versus legality.
"Tony and I only have our passports to prove we are Gordon Glassford and don't use this identity in our daily lives at all, but our children have had to use it everywhere, like school/university qualifications, bank accounts and driver licences yet don't identify with it if they were introducing themselves to someone.
"Tony and I did not think it was our choice to make for our children and the world was much more relaxed about identity when they were born. Now everything has become very tight and rigid.
"I would not change my identity again like I did when I got married, so if forced to make a decision would pay the $180 and have my name formally changed to just Glassford. This would be the simplest answer for me and all I would have to change is my passport, all other documentation is already in this name and I don't have to ask people to recognise me under a new identity again, and remember in my bones I'm actually a Sinnamon.
"I married the story — I wasn't born into it, so who I am in relation to it is pretty irrelevant."
Their dream would be to go to Glassford, in south Lanarkshire in Scotland, to see where it all began.
While acknowledging his family's colourful past, Mr Glassford is very much focused on the farming future through regenerative agriculture.
The science and the practicalities behind it made sense to him. He thinks his ancestors would approve.
"The whole aim is to let the soil rejuvenate, and let the animals do the rejuvenating.
"I've always tried to look after the land. The most important part of farming is to look after the topsoil.
"It's what we grow the plants in, it's what we grow the meat on."