From birth, the Coke family were close to the Windsor clan, with her father serving as King George VI's Extra Equerry and later, her mother would be made a high-ranking Lady-In-Waiting to Queen Elizabeth.
From the age of two or three, she says she remembers the young Princesses coming to play, the trio descending on the beach nearby the family home to build sandcastles. When Prince Charles was young, he would be sent by the Queen to Holkham for weeks on end when he contracted contagious childhood illnesses like chickenpox to prevent Her Majesty being infected.
MARRIAGE AND MARGARET
1950 saw Anne turn 18 and thus she was dutifully 'launched' onto the British social scene with an eye to nabbing a husband, even being dubbed "debutante of the year" by Tatler. She soon caught the eye of Johnnie Spencer, the heir to Althorp estate and the future Earl Spencer.
For the stunning young woman, it was love at first sight and soon the couple became unofficially engaged.
However, they conducted their romance both at Holkham and at the London home of Lady Ruth Fermoy — and this is where things started getting tricky.
Lady Fermoy was not only a close friend of the Queen Mother but had a 15-year-old daughter named Frances. The society matron invited Anne to bring her lovely chap around for a drink and who happened to be there? Teenage Frances of course.
"Suddenly, I could see that Lady Fermoy was pushing Frances like mad. 'Do you like tennis, Johnnie? Oh, Frances adores tennis, don't you, Frances? Do you like swimming? Oh, Frances just adores swimming, don't you, Frances?'" Anne said according to royal biographer Tina Brown.
The inevitable happened: Johnnie fell for Frances when she was older, dumping Anne, which was a "brutal rejection". (Supposedly Johnnie's father had warned him off wedding Anne, claiming her family had "mad blood" based on the fact that two distant relatives had been put in a mental asylum.)
"I was devastated when Johnnie dropped me. I was sent off on the QE [cruise liner] to America ostensibly to sell Holkham pottery, but really to get over it. I sometimes wonder if I wasn't rich enough."
(Johnnie and Frances would marry in 1954 and go on to have four children including Lady Diana Spencer. Lady Fermoy would disastrously and historically play matchmaker again, partnering up with the Queen Mother to marry off Lady Diana to Prince Charles.)
Then, in 1955, came Colin Tennant.
The second son of the 2nd Baron Glenconner, Colin had the perfect pedigree on paper — Eton followed by Oxford and was very wealthy to boot. They met at a debutante party at the Ritz in London and soon started dating. In 1956, they tied the knot at Holkham.
Unfortunately, one thing the bride-to-be had not been told was Colin's history of mental health issues, including suffering from two nervous breakdowns.
Anne recounts that years later she asked him: "Why had he picked me to marry … when he had lots of more sophisticated girlfriends. He replied: 'Well, I knew that you'd carry on, you'd never give up'."
The Glenconners' wedding was to offer another spooky instance serendipity. Princess Margaret and the Queen Mother both attended the nuptials. And who should be chosen to photograph the big day? Antony (Tony) Armstrong-Jones aka the Princess' future husband, the Earl of Snowdon.
MUSTIQUE
In 1957, Colin (he would only assume his title in 1983 when his father passed away) paid $81,000 for the entire, tiny Caribbean island of Mustique. The newlyweds built a very basic hut, with no electricity or running water, but soon the lure of this tropical paradise grew.
When Princess Margaret wed Tony in 1960, Colin suggested they 'pop in' to Mustique on their honeymoon.
"It's very primitive but it has magical beaches. Anne and I will be there, living in our hut, and we won't bother you at all," he is reported to have said.
Thus, according to reports, what should appear on the horizon one day but the royal yacht Britannia, carting Margaret and Tony around the Caribbean on their honeymoon.
Later, a sailor was soon sent to ask the Glenconners to dine on-board.
"I sent a message back saying we'd love to,' Anne has previously said, "but that as we hadn't had a bath for a month could we possibly have a bath first? Our hut was very primitive — no hot water, electric light or anything like that."
At some stage, Colin offered to give Princess Margaret a piece of land on the island as a wedding present and later rolled up at Kensington Palace with a map and the requisite documents to hand over the property.
In 1968, Margaret visited Mustique again but it would take until 1972 for the construction of her own home on the island to be completed. (A huge crate of furniture was shipped over from London.)
Named Les Jolie-Eaux ('The beautiful waters') the house was a five-bedroom villa set on nine acres that would increasingly provide a haven as Margaret's marriage to Tony crumbled.
For more than three decades, essentially until her death, Margaret would descend on this slice of secluded paradise every February and again in late Autumn, soaking up the sun, giving in to her hedonistic urges and reigning over the nascent social scene. (Colin had sold other plots of lands to wealthy and titled chums, helping create the exclusive private haven that it is today and which helps explain why it is the Cambridges' favourite holiday spot.)
As her biographer Theo Aronson notes: "More than any other place, Mustique came to symbolise, in the public mind, Princess Margaret's sensual and sybaritic nature."
However, the island would also come to be intimately associated with one of biggest scandals of Margaret's life.
LADY-IN-WAITING
In 1971, the closeness between Anne and Margaret was formalised when the HRH asked the mum-of-five to be one of her ladies-in-waiting. Their bond went far beyond that of mistress and underling, with the Princess openly talking about her faltering marriage and her husband's acts of petty aggression.
Anne writes: "She told me, for instance, that she no longer opened her chest of drawers — she got her maid to do it instead — because Tony had developed a habit of leaving nasty little notes inside. One of them said: 'You look like a Jewish manicurist and I hate you'."
Years later, in 1990, when Colin sold the Glenconners' London home without telling his wife, it would be Margaret who would come to the rescue, welcoming Anne to live in Kensington Palace with her for a whole year.
Theirs was a true friendship with Margaret turning up at Anne's far-from-palatial country home with her own kettle enjoying long walks and fireside chats. ("'What about another little drinkie-winkie?' she'd say," Anne has recalled "and Colin would appear with another round of drinks — whisky for her, vodka tonic for me.")
MATCHMAKER
And then, one weekend in 1973, everything changed, irreparably.
The Glenconners were having the Princess and assorted friends up to their vast Scottish estates called Glen. Down one man (the goal was always to have even numbers of each gender), a London society matron suggested they invite 25-year-old Roddy Llewellyn to make up the numbers. His father had won gold at the Melbourne Olympics.
Colin drove to Edinburgh to pick up their guests but things soon went haywire.
"They didn't return for hours," according to Anne's book.
"Forewarned by her protection officer, however, I was outside, ready to greet them, when the car pulled up. In the back, Princess Margaret and Roddy were more or less holding hands.
"Colin explained that they'd met him off the train and gone for lunch at a bistro. The princess and Roddy had immediately clicked, even though he was 17 years younger. She'd then whisked him off shopping to find some tight swimming trunks — which my son described as 'budgie smugglers'.
"I said to Colin, 'Oh, gosh, what have we done?'"
By the weekend's end, it was blatantly apparent that despite being 17 years Roddy's senior, the duo were smitten.
Over the course of the next three years, Margaret and Roddy's relationship would face ups and downs, with him struggling with life as a not-quite-official royal plus one. While he was introduced to the Queen Mother, her sister the Queen was not impressed and according to one biographer referred to "my sister's guttersnipe life."
However, one place the couple could be more open about their relationship? Mustique.
MEDIA MELTDOWN
The Princess and Roddy used to travel together under assumed married names. In 1976 they made at least their third trip together to the island to spend time with each other.
Unfortunately, there was someone else already there: A photographer had posed as a holiday-maker with his wife.
While the Princess rarely ventured out in public, preferring to entertain at home, she occasionally had drinks at the Beach Bar of the Cotton House Hotel. It was there that the sneaky snapper got the picture he had come for — Margaret with her toyboy lover. Anne's brother and sister-in-law also being in the shot but they were cropped out to make it look like it was only the lovers.
Days later, the photo found its way onto the front page of the News of the World and all hell broke loose.
Despite the fact he had been having a long-term affair with his assistant Lucy Lindsay-Hogg, Tony played the wounded husband and used the release as an excuse to paint Margaret as the villain in their marriage. He promptly moved out of Kensington Palace and the couple's divorce was duly announced.
The story dominated headlines and was reported on TV and radio.
Interestingly, despite the onslaught, Margaret and Roddy's romance would continue for a number more years before he would fall in love with another woman.
FAMILY TRAGEDY
While Anne might have had had a front-row seat to some of the royal family's more saucy moments, her life has been marred by tragedy. Their oldest son was a heroin addict who overcame his addiction before dying of hepatitis C, the second died of AIDS and the third was left paralysed after a motorbike crash.
However, there was more heartache to come. In 2010 Colin passed away. When the extended family gathered to hear his will read, they were aghast to find out he had left his vast holdings and estate to one of his servants. It would take seven years of legal wrangling before Anne's grandson Cody, the rightful heir to the title and fortune, was awarded 50 per cent of the estate.
In 2002, Anne faced the loss of her great friend, Princess Margaret.
"Her death in February 2002 left me feeling extremely sad, even lost," she writes.
"We'd been such good friends, and had spent so much of our time together, that her absence left an enormous hole in my life."