They were also criticised for eschewing joining the Queen at her beloved Scottish estate Balmoral. (Favourite activities: Long walks across midge-infested moors and fly fishing. When Diana once made an early exit she offered only two words of explanation, according to her biographer Tina Brown: "Boring. Raining.")
Despite all of this, last summer might seem light a cakewalk compared to the coming months, which are shaping up to be a testing, tricky time and could have long lasting consequences for their brand, business and family relations. (So, you know, no pressure or anything.)
Last week Prince William, Prince Charles, Camilla Duchess of Cornwall and Princess Anne all returned to real world official duties, all looking very serious and drenched in hand sanitiser. With things slowly inching towards something resembling normality, the question is whether we will see Harry and Meghan step out in Los Angeles for some sort of quasi-official outing.
While plans to debut their new charity initiative Archewell have been pushed to 2021 (plus they are facing a few wee pesky issues with the associated trademark application) they must still surely want to kick start their professional lives in the US. (If nothing else, they are both driven by an authentic desire to do good and twiddling their thumbs playing Animal Crossing and catching up on Normal People just does not seem their style.)
Unshackled from the constraints of royalty, they are now free to be as political and voluble as they want. They have a global platform, the challenge they now face is how to use it at this time when there is a global conversation going on about racism and power.
And herein lies the rub: Given the growing, powerful momentum of Black Lives Matter they will surely have to (and more importantly want to) engage with this movement and to play a part in this moment. However, the pressure they will face to pitch their engagement perfectly will be immense.
If for some reason they decide not to be a part of this any further (Meghan has already given a powerful graduation speech about addressing racism) or for some reason they botch their handling of it, the consequences would be disastrous as they go about building their public lives in America.
Summer itself, and the usual toff pleasures of the warmer months, also pose a series of thorny dilemmas.
To start with, the polo season is set to start back up soon. Charity games have been a source of fundraising revenue for Harry's charity Sentebale and he and William have previously played in the King Power Royal Charity Polo. Assuming that Harry is able to travel, he would be criticised if he stayed away or did not take part.
That said, returning to Blighty and wading back into a reported family feud does not sound particularly appealing. Do the Sussexes, after a wildly turbulent and most likely draining year, really want to have to deal with being on very public display with the Cambridges with a veritable army of cameras trained on them?
If they go back to the UK, this in turn would present them with another quandary – to go to Balmoral or not. For years Harry was famously close to his grandmother (who else could have gotten Her Majesty to take part in a cheeky spoof video with the Obamas?) and the Sussexes video called the 94-year-old for her birthday in April. Do he and Meghan really want to spend their downtime disentangling Archie from fishing lures and spraying themselves with Aeroguard?
No matter how Harry and Meghan deal with each of these situations, one thing is glaringly apparent. They cannot stay away forever.
Practically they have a number of ties with the UK in the form of both of their various patronages and the charities they work with. They have strenuously maintained they will continue with those roles (plus Harry is president and Meghan is vice-president of the Queen's Commonwealth Trust) meaning at some stage they will have to wing their way back across the Atlantic and plunge themselves back into the family, public and media maelstrom they only temporarily left behind.
(If nothing else they are paying more than $32,000 every month for Frogmore Cottage, a five-bedroom house that has pretty much sat empty since November last year.)
When Harry and Meghan walked out of Westminster Abbey in March during their final official outing as working members of the royal family, it was in a blaze of headlines about the positively Arctic dynamic with William and Kate, Duke and Duchess of Cambridge.
While the Sussexes might have left the UK behind, when they return, and they will have to, they will walk right back into the same narrative.
While Meghan beamed her megawatt smile during that outing, Harry remained stony faced and looked positively miserable. Given their return, at some stage or another is guaranteed, I wonder if this was the politic and strategic move.
The crux of all of which is to say that while Harry and Meghan might have traded palace life to live in Tyler Perry's $30 million Beverly Hills bolthole, they have far from left their problems behind.
They say you can't escape your past or your family. This northern summer, Harry may very well be about to learn that the hard way.
