As firefighters on tall ladders doused the flames the evacuees, many dazed and in shock, gathered with curious onlookers along Knightsbridge to stare at the building.
London Fire Brigade sent 20 fire engines and 120 officers and firefighters to the hotel, which backs onto Hyde Park, along with several ambulances.
Witnesses told how at one stage several of the Mandarin's guests returned inside the building, thinking they were safe - only to be ordered out again.
Simon Cockshutt, a retired solicitor, said: "People starting coming down the external fire escape as the thick black smoke billowed above them. At one point the smoke stopped and some of the guests went back inside. A few minutes later it started again and they started leaving again."
Cockshutt was among a group of 60 former pupils from St George's School in Weybridge who were evacuated from the next door Royal Thames Yacht Club midway through their over-60s school reunion.
Gerry Johnson, an investment banker, said: "Suddenly we were told to get out. There's a suggestion the fire started on the roof of the Mandarin, where they have been carrying out building work."
The owners of the international hotel chain recently announced the end of renovation work "designed to confirm this historic hotel's position as one of the best in the world".
The hotel is between Hyde Park and the Knightsbridge shopping district, which includes Harrods.
It is also directly next door to One Hyde Park - one of London's most expensive buildings - where a five-bedroom apartment was on the market in 2015 for £75 million ($142m).
The scenes in Knightsbridge brought back stark memories of last year's Grenfell Tower fire, which claimed the lives of 72 people and is now the subject of a major public inquiry.
Penelope Evans, 50, from Battersea, south London, was shopping in Harrods when she noticed the thick smoke as she was leaving.
"I saw all this black smoke coming from the sky, it was covering everything and the smell was horrendous," she said.
"It was like people were in shock, everyone immediately just started leaving the street and the area.
"After Grenfell, I feel like people are a lot more on edge about these things, people just wanted to get out."
London Ambulance has sent its hazardous area response team (Hart) to the hotel, but there have been no reports of injuries and paramedics have yet to treat any patients.
Hart are specially trained medics who provide life-saving medical care in hazardous environments including complex extractions and multiple casualty incidents.
The 12-storey Mandarin Oriental has previously been used by the royal family and the Queen hosted an event there on the eve of Prince William's wedding to Catherine Middleton in 2011.
It was due to host another royal event tonight - the 50th Anniversary Dinner of the Blues and Royals Club - to be attended by the Princess Royal.
But a Buckingham Palace spokeswoman told the Telegraph the celebration had been cancelled.
A hotel spokesman said it had no further details to share.
