But, as we discovered when we explored the superb museum in Colchester Castle, its origins go back much further than that. As Camulodunum, principal settlement of the powerful Trinovantes tribe, it had some claim to being the capital of Celtic Britain.
The Romans certainly made it their capital and built a vast temple to the Emperor Claudius, so magnificent it attracted worshippers and tourists from all over Europe, until it was destroyed during the great rebellion by Queen Boudica and the Iceni and Trinovantes tribes.
William the Conqueror also recognised it as a key town and Colchester Castle, built on the foundations of the great temple and using a lot of Roman brick in its walls, was one of the first stone castles he ordered built to consolidate his realm.
With so much history it seemed entirely plausible that Colchester had been around long enough to have inspired three nursery rhymes.
And, sure enough, as we stood outside the quaint old St Helen's Chapel, a Saxon place of worship now used by the Greek Orthodox Church, Baalham explained that merry old King Cole was actually a Roman decurion called Coel who ruled Colchester - the name means Cole's Castle - a couple of thousand years ago.
Coel was also, he added, father of Helena, wife of the Emperor Constantius, and mother of Constantine the Great, the emperor who converted the empire to Christianity, thus earning sainthood for his mother.
Of course, as with anything to do with nursery rhymes, this is all hotly disputed, but the people of Colchester long ago adopted Helena as their daughter. The town's coat of arms, dating from 1413, honours her life by depicting the true cross on which Jesus was crucified, the three iron nails with which he was fastened and the bodies of the three wise men, all of which Helena discovered during a pilgrimage to the Holy Land.
The second rhyme seemed less controversial. As we walked through the Dutch Quarter, named after the Flemish weavers who settled there in the 16th century, Baalham pointed out a plaque high up on a lovely old brick house proclaiming that Twinkle, Twinkle, Little Star was written there in 1806 by Jane Taylor, for a book called Rhymes for the Nursery, produced with her younger sister Ann.
But, like I say, anything to do with nursery rhymes seems inevitably to provoke bitter debate. I've since discovered that the town of Lavenham in Suffolk reckons that Jane Taylor wrote Twinkle, Twinkle there.
Meanwhile, back in Colchester, as we wandered alongside a stretch of the city's ancient Roman wall, past what is said to be the best preserved Roman gate in Britain, Baalham said he was about to show us the link to the third nursery rhyme.
Just inside the wall, as you'd expect, is St Mary's-at-the-Wall, dating to 1206 and notable as the spot where 23 Protestant martyrs were burned to death during the reign of Mary I.
It's also, according to Baalham, widely thought to be the place where Humpty Dumpty originated.
"The story goes that during the Civil War, when the Parliamentary forces were besieging the town, a fat Royalist called One-Eyed Jack Thompson took a canon up into the church tower and caused mayhem by firing at the attackers.
"The Roundheads got a huge canon from the Tower of London, fired it at the tower and blew him out, so Humpty Dumpty came down and all the king's horses and all the king's men couldn't put him back together."
This sounded amazing - but Baalham held up his hand to add more. "Unfortunately, it's not true. A local historian who researched it found the first mention of that was in 1950. It seems as though it was made up then. But the idea has stuck and we can't get rid of it. In fact, when I went for my guide's badge I was informed that if I told that story I would be failed.
"So now I tell people the story of One-Eyed Jack, which is true, but make it clear that it's got nothing to do with Humpty Dumpty."
The tower was later rebuilt, but in 1978 the church closed, the bell in the tower was removed, and two years later it opened as the Colchester Arts Centre.
Looking up at the tower, silhouetted dramatically against the darkening evening sky, it was easy to imagine a chubby Royalist nicknamed Humpty sitting up there and giving the Roundheads a hard time until they brought him crashing down to earth. It's the sort of thing that could easily have inspired a wee poem.
Funnily enough most of the sources I've looked at since seem to assume the old nursery rhyme does in fact come from Colchester. But please don't ring me up with your version. I make no judgment on any of it.
CHECKLIST
Getting there: Emirates flies three times daily from Auckland to Dubai, with direct connections to its daily services from Dubai to six airports in the United Kingdom.
Where to stay: The marvellous old Rose and Crown is said to be the oldest pub in the oldest town in Britain.
Where to eat: Try the Lemon Tree restaurant, which not only has great food but also a bit of Roman wall in its dining room.
Further information: See visitessex.com and visitbritain.com.
Jim Eagles visited Colchester with help from Emirates and Visit Britain.
