Princess Diana, pictured on June 1983. Photo / Boris Spremo, Toronto Star via Getty Images
In case you missed it: This was one of Life & Style's biggest stories from 2015.
Saturday Aug 15, 2015
When messages began to filter through from Paris to the Royal Family's Highland retreat at Balmoral during the small hours of August 31, 1997, that Princess Diana had been involved in a serious car crash, the Queen could barely believe what she was hearing.
At first it was thought that, though the car crash in the Pont de l'Alma tunnel was serious, Diana had not been killed.
Taken behind the scenes at Buckingham Palace, they show Diana and the Queen walking side by side down a corridor in the aftermath of the ceremony.
Yet despite the joyous occasion, there is little evident warmth between the two women or even a flicker of happiness on either face - a glimpse, perhaps, of their underlying anxieties and the great emotional gulf between two such differing personalities.
So what did the Queen truly make of her daughter-in-law? The answer, I discovered while researching an in-depth new biography of our monarch, is utterly intriguing.
On September 9, the Queen will surpass the 63 years and seven months that her great-great-grandmother Victoria was queen - making her the longest reigning monarch in British history. It seems unimaginable that anything could cast a cloud over her rule.
It is difficult now to realise how badly mauled the monarchy was just 18 years ago when - after years of marital feuding and scandal between the Prince of Wales and his estranged wife - the news of Princess Diana's death traumatised the country.
When Lady Diana Spencer first visited Balmoral, aged 19, she charmed all the Royals and the Queen especially. Her father, Viscount Althorp, had served as an equerry to the Queen between 1952 and 1954, and to George VI for the two years before that.
Her grandmother, Ruth, Lady Fermoy, was a friend and lady-in-waiting to the Queen Mother.
The family lived at Park House on the Sandringham estate and the Queen had seen Diana growing up: her elder sister Sarah was a former girlfriend of Prince Charles and the other sister, Jane, was married to Robert Fellowes, the Queen's assistant and later her private secretary.
"She is one of us," the Queen wrote to a friend. "I am very fond of all three of the Spencer girls."
At Balmoral in 1980, Diana joined in with the after-dinner games, laughed at Prince Philip's jokes, fell into bogs and got wet, and said all the right things.
She was accepted warmly into the royal circle.
Watch: Rare photos from wedding of Charles and Diana
But what the Queen did not recognise was the teenager's shallowness.
Diana was naive and not given to looking beyond the moment. By contrast, the Queen always had one eye on the future, even as a child.
When she was told, aged ten, that her uncle, Edward VIII, had abdicated and her family, with her father as the new King, must move into Buckingham Palace, she asked at once: "What, forever?"
Diana was dazzled by the romance of her own situation. With a magnificent oval sapphire on her engagement finger - the same one, of course, now worn by the Duchess of Cambridge - she felt she had, in her words, "caught the big fish".
It was not until she found herself with a permanent police escort and was living in the Palace's former nursery suite on the second floor with all her old freedoms curtailed that she began to consider the reality of life as a Princess.
The Queen made a great fuss of her future daughter-in-law, trying to demonstrate that she was interested in Diana for her personal qualities and not just for what she represented, as the wife of the heir to the throne.
But Diana ran out of things to say to her. Understandably nervous, she didn't want to have lunch on her own with "Brenda" - her nickname for the Queen, taken from the satirical magazine Private Eye - and made excuses, even inventing non-existent friends to avoid the invitations.
The Queen could see the much younger woman was anxious, but had no inkling of her emotional problems or knowledge of issues such as bulimia, the eating disorder that would plague Diana for years.
The wedding at St Paul's in July 1981 was a royal occasion on a scale never seen before, not even for the Queen's Coronation.
Every detail was magnified beyond imagination: the palace ball before the ceremony was the most lavish in more than half a century, with just about every European royal, as well as America's First Lady Nancy Reagan, and a raft of prime ministers and Commonwealth leaders on the guest list.
On the day itself, a vast crowd gathered in the Mall to see Charles and his bride appear on the balcony. Listening to the roars, Diana said to her husband: "They want us to kiss."
They did, and the moment was shared by a worldwide TV audience of 700 million.
That night, the Queen attended a party at Claridges, where video screens replayed the vows - watched by the monarch, the First Lady and Princess Grace of Monaco, seated together on a circular sofa.
But in the coming months, the Queen was troubled by the unflagging media attention. The Press simply couldn't get enough of Diana and all other royal business paled by comparison - especially when news broke in November of her pregnancy.
Pointedly, during that year's Christmas broadcast, the Queen did not dwell on the wedding celebrations, but singled out what she described as "a very different scene", a garden party at the Palace for 3,500 guests with disabilities.
Concerned that Diana was not coping well with all the attention, the Queen instructed her press secretary to invite all Fleet Street's editors to a meeting.
In an almost unprecedented move, she appealed to them to rein in their coverage, speaking to them individually or in small groups.
This plea from the heart worked - but not for long. It didn't help that Diana resented any shift of focus away from her.
The christening of her first child, Prince William Arthur Philip Louis, for instance, fell on the Queen Mother's 82nd birthday, so that she and not the baby was the centre of attention.
Diana later complained she felt "totally excluded" and William, sensing his mother's mood, cried throughout the christening.
Two years later, after a difficult second pregnancy that left her tired, overwrought and thoroughly miserable, Diana told friends she was not "made for the production line".
The Queen sympathised, but still felt sure her daughter-in-law would learn to adapt to royal life: the Princess's relaxed, informal style in public, after all, meant her popularity was unparalleled.
In later years, the Queen would reproach herself for not seeing how much strain the Wales's marriage was under.
She knew she was not a tactile mother: like many aristocratic parents of her generation, she had delegated much of the childcare to nannies and to her own mother.
Though never giving way to mawkish regrets, she sometimes blamed the disintegration of not only Charles's marriage, but Anne and Andrew's as well on her own remoteness when the children were growing up.
The Prince of Wales must have felt it, too, because when he needed to pour out his heart about his troubles with his wife, it was to the Queen Mother he turned and later to Camilla Parker Bowles, never to his own mother.
The depth of Diana's unhappiness became plain only when she collaborated with journalist Andrew Morton on a book that became "a catalogue of marital grievances", as one historian called it.
She gave off-the-record interviews and authorised her friends and family to speak to Morton.
When the book appeared, sparing no detail, the Queen clung to the delusion that Diana could not have been involved.
The Princess lied to the face of Palace private secretary Robert Fellowes, her own brother-in-law, and denied all complicity. The Queen believed her.
But a week later the secret was out, when she pointedly visited one of the book's named sources, Carolyn Bartholomew. Diana was a proven accomplice.
Fellowes did the honourable thing and offered the Queen his resignation. She refused it on the grounds that he was not the one guilty of misleading her.
Six days after the story broke, Diana stood beside her mother-in-law on the Palace balcony after the Trooping the Colour, as if nothing was amiss.
But the facade had to crack. At Ascot the following week, Prince Philip snubbed Diana in full view of everyone in the Royal Enclosure.
Even then, the Queen believed in diplomacy. She ordered a six-month cooling-off period to let tempers die down.
But she had again failed to understand why the Princess behaved in such an erratic, provocative way.
Charles's patience snapped when he arranged a shooting weekend at Sandringham with his sons, who were then at prep school, only to be told that Diana had taken them to Windsor by herself.
Ranting on the phone to his mother about his wife's latest calculated outrage, Charles forgot himself and shouted down the line at the Queen: "Don't you realise? She's mad, mad and mad!"
Diana did nothing to dispel the accusation when she started hinting darkly that Palace courtiers were conspiring to smear her by using the secret services to eavesdrop on her private conversations.
The Queen dismissed this as nonsense, but refused to allow the family to discuss the Charles and Diana situation openly.
Princess Margaret, the Queen's sister, confided in friends that the topic was so loftily off-limits that no guest would dare refer to it.
Like her own mother, the Queen has always coped with troublesome emotions by keeping the various difficulties of her reign in airtight compartments and never confronting the unpalatable.
But the marriage breakdown could not be ignored forever. On December 9, 1992, Prime Minister John Major told the Commons that "with regret, the Prince and Princess of Wales have decided to separate".
The Queen was at Wood Farm on the Sandringham estate with only a handful of staff when the announcement came.
It was a poignant setting: here, in 1919, in this redbrick house hidden from view at the end of a tree-lined drive, her 13-year-old uncle, Prince John, had died of an epileptic fit.
Instead of watching the statement to Parliament, the Queen did what she often did when agitated, and took her corgis for a walk through the wintry woods and over the ploughed Norfolk fields.
When she got back, she dried the dogs off - and almost immediately took them out again, dressed in her usual country garb of wellington boots, Loden coat and headscarf.
As she returned for a second time, a senior member of staff approached to offer his condolences. The Queen replied briskly "I think you will find it's all for the best", and walked out once more into the drizzle.
The next five years brought little respite. Especially upsetting was a biography of Charles by the broadcaster Jonathan Dimbleby, which presented his relationship with his parents as remote: the Queen was portrayed as cold; the Duke of Edinburgh as a bully.
The Queen was so concerned at the endless criticism of her and her family that she became convinced, in May 1995, that the country would turn against the Royals during the 50th anniversary celebrations of VE Day - and that the crowds would stay away from the Palace.
Throughout the early morning, she kept looking anxiously out of the window, to check whether her subjects were waiting to see her.
To her unspeakable relief, by the time she made her balcony appearance with her sister and their 94-year-old mother, the Mall was packed.
"Her Majesty was thrilled," a member of staff revealed later. "When she went on to the balcony she remained stony-faced for fear of showing too much emotion. She was actually close to tears."
The crowds that gathered outside the Palace two years later were in a very different mood. As days passed after Diana's death and there was no word from the Palace, they were veering perilously close to becoming a mob.
Many people condemned the Royal Family vociferously for staying in Balmoral, instead of returning to London, and for refusing to fly a flag at half-mast over Buckingham Palace.
The Queen was bewildered by these criticisms. The business of the flag was mere protocol: she was not in residence, so the flag was not flown.
Far more important, she wanted the family to stay in Scotland to give her grandsons a chance to absorb the shock of their mother's death as far as possible from the public eye.
Her first priority was to protect them.
On the morning that Diana died, Charles broke the awful news to his sons before the whole family went to church at nearby Crathie. After that, the boys were encouraged to mourn in private.
The Queen saw prime minister Tony Blair's public statements were much better suited to the national mood, but his approach was not one she could adopt.
She finally returned to London on September 5 and was driven straight to the Palace where, with Prince Philip at her side, she left the safety of her car and went to mingle with the throng beside the flower-covered railings.
Dressed in black, she walked along the line of mourners in total silence until an 11-year-old girl handed her five red roses. "Would you like me to place them for you?" asked the Queen.
"No, Your Majesty," replied the girl. "They are for you."
An aide recalled: "You could hear the crowd begin to clap. I remember thinking: 'Gosh! It's all right.'"
By the time she made her live broadcast that evening, the Queen was more her usual self.
She addressed the nation "as your Queen and as a grandmother" and paid tribute to Diana: "She was an exceptional and gifted human being. In good times and bad, she never lost her capacity to smile and laugh, nor to inspire others with her warmth and kindness.
"I admired and respected her... especially for her devotion to her two boys."
Seven years later, as she opened the Diana Memorial fountain in Hyde Park, she remembered her shock at learning the news of her daughter-in-law's death.
"Certainly the days that followed are etched on my memory as we as a family and nation came to terms with the loss, united by an extraordinary sense of shock, grief and sadness."
During those years, she had become increasingly frustrated by the cat-and-mouse game that Charles played with the Press and his mistress.
The strong, loving relationship between the Prince and Camilla had long been public knowledge, but he did not seem to dare to make it official by marrying her.
Privately, the Queen felt her son's indecision was ridiculous - particularly with regards to the succession. "What if I fell off my horse?" she demanded of one relation. "The situation has to be resolved."
It was, at Christmas 2004. Charles worked up the courage to broach the matter, and his mother happily gave them her blessing.
At the wedding reception on April 9, 2005, the Queen made a rare public comment on family business.
Comparing the many obstacles that Charles and Camilla had encountered to the Grand National racecourse, she told guests: "They have overcome Becher's Brook and the Chair, and all kinds of obstacles.
"They have come through and I'm very proud and wish them well. My son is home and dry with the woman he loves."
It was a very long way from the darkest point of the Nineties, when the Queen felt she had failed Charles and Diana - and, one day, had turned to her mother in mock despair and asked where it had all gone wrong.
The Queen Mother had been playing one of her customary games of patience. She looked up from her cards and said: "Don't worry. It will be all right in the end."