Four years ago, his cargo ship, the Maersk Alabama, was hijacked by Somali pirates in the Indian Ocean, handing the pirates the modern equivalent of a Spanish galleon full of bullion.
Not because of the cargo on board, which included food aid for Rwanda, but because all but one of the 20 crew were Americans - a jackpot in terms of high-value hostages. If the shipping company wasn't prepared to pay a ransom, there'd be no shortage of buyers on Somalia's mainland, home to the al-Qaeda-linked al-Shabaab movement.
For a time, it looked like becoming America's worst hostage crisis since the 1979 Tehran embassy siege, a serious test of President Barack Obama's mettle during his first months in the White House.
Instead, it turned into a tale of all-American heroism, as members of the crew, hiding below deck, turned the tables by overpowering the pirate leader and taking him hostage. Phillips, who was being held at gunpoint on the ship's bridge, then allowed himself to be taken as collateral for a "prisoner swap", figuring that one US hostage was better than 20. Finally, as the pirates tried to take Phillips to the Somali coast, two US Navy warships blockaded them, resulting in a tense three-day stand-off.
The drama, covered in real time on US television networks, made the seamen the toast of America. Phillips' wife, Andrea, got a call of congratulations from a relieved Obama, while Phillips went on to co-author a best-selling book, A Captain's Duty.
And now, the hostage story with the Hollywood ending has had the Hollywood treatment. Captain Phillips, starring Tom Hanks as the hero, is released on October 24 and has already received enthusiastic reviews. So has the chip finally been torn from Phillips' epaulette about the way the world sees merchant seamen?
"It is a good film, yes, and it does portray a captain in peril. I didn't really care who played me ... but Hanks does a very good job ... He is a very nice gentleman, a very average Joe."
It was the sense of two different worlds colliding that made the story a draw for British director Paul Greengrass, the ex-documentary maker who was also behind the acclaimed United 93, about the hijacking of one of the four 9/11 airliners.
He was intrigued by how the open ocean, a place most people never go, is where the lifeblood of modern capitalism comes in contact with the chaos of Somalia, where decades of anarchy have led to the rebirth of a medieval barbarity like piracy.
The film touches on the root causes of the piracy crisis, which, at its peak in 2011, saw some 700 sailors in captivity. Eyl, the pirate port from where Hanks' captors set out, is vividly brought to life as a modern-day Hispaniola, a lawless, dirt-poor place where buccaneering is the only decent living. The script also raises the question of whether Somali fishermen were driven into piracy by foreign trawlers who plundered the fish stocks in their unpoliced waters.
But there is no Oliver Stone-style lecturing, and the pirates, who are played in the film by untrained actors from the "Little Mogadishu" district of Minneapolis, are a terrifying presence, utterly convincing.
And I should know. Five years ago, I was kidnapped by pirates in Somalia myself while reporting for the Sunday Telegraph. It is perhaps a testament to the Somali actors' abilities that my instinctive reaction on watching the film was to want to see them dead.
Phillips, now 58, is not enamoured with them either. Just like merchant seamen, he says, pirates don't get the PR they deserve. All that Pirates of the Caribbean stuff, with Johnny Depp prancing around like a dandy, is romantic twaddle, he says. Pirates are just cold-blooded persecutors of ordinary sailors like him, whose job is already quite dangerous enough.
"They are very cruel people, preying on merchant seamen who already have to deal with all kinds of other hazards, like storms or fires on board, all of which can be a death sentence," says Phillips.
All the same, it was a certain swashbuckling instinct that drew Phillips to the sea in the first place. After a wayward youth in Irish-American Boston, by 1974 he was earning a living as a cabbie, his life going nowhere, when a man in an expensive leather jacket clambered into his cab around 10 o'clock one morning, telling him: "I want booze and I want broads." Upon being handed a US$5 tip for the US$5 ride to the nearest fleshpots, Phillips asked how his fare earned such a handsome living. "I'm a merchant mariner," came the answer. "We carry cargo in ships."
The cargo bit sounded dull, but the off-duty "booze and broads" sounded fun, so off Phillips went to Massachusetts Maritime Academy.
In the following decades he fell in love with his day job too, travelling the world on three-month shifts, but also eventually raising a family at a 19th-century farmhouse back in Vermont.
The trip when he was hijacked, sailing from Oman to Mombasa, should have been routine: the inevitable row with his wife just before he left for sea, compensated for by the usual phone calls home, where he'd cheer her up with Barry White impressions.
But by March 2009, the Indian Ocean was no longer the relaxed, easy place he'd sailed through many times before. Hijackings had been skyrocketing since the previous year, with the pirates ranging hundreds of kilometres out to sea in so-called "mother ships". Merchant vessels were beginning to kit themselves like floating fortresses, with high-pressure hoses, anti-piracy fences and audio guns that transmit a deafening sonic ring. On Phillips' own vessel, they also had axes and lead pipes stockpiled - not much use against AK-47s.
As Phillips puts it in his book: "It was like a lion and a herd of wildebeest on the African plain. You just hoped there was safety in numbers, because if the lion chose you, you were going to have a very, very bad day. And just as the lion looks for weakness - the slow, the lame, the young - pirates zeroed in on ships that looked defenceless."
His crew was also relatively unusual in being almost entirely Americans, rather than the Third World nationals who account for most of the world's one million commercial sailors these days.
Phillips' vessel was the first US-flagged ship to fall victim to piracy since the 19th century.
The first few days of the voyage went peacefully but anxiously, Phillips practising anti-piracy drills and scanning the latest hijack reports: 39 attempts in a single week. Then, one lunchtime, three fast-moving blips appeared on the radar, heading straight for the Alabama. It was three pirate skiffs, full of men with machineguns.
Phillips sped up and headed into heavier seas, slowing the pirates down with the waves in his wake. He also faked a message over the radio from a US Navy ship, saying a helicopter gunship was just five minutes away. To his delight, the pirates gave up. But the next day, another skiff approached. This time, the sea was dead calm, preventing the Maersk Alabama from outrunning them.
Soon, they were peppering the ship with bullets, Phillips trying to keep them at bay by firing distress flares. The next thing, they had hooked a grappling ladder to the ship's side and swarmed on board.
So began Phillips' acquaintance with the Leader, Tall Guy, Musso, and Young Guy, whose eyes lit up when they realised they had a ship full of Americans. The question for the pirates, though, was how to find them all. While Phillips and two others were on the bridge, the rest of the crew had hidden in the ship's 152m-long hold, a labyrinth of passageways, cargo areas and engine works.
Even in normal times, stowaways can hide there and simply never be found. And on this occasion, the area below deck was pitch dark after the crew disabled the emergency lights.
Despite being ordered at gunpoint to summon the rest of the crew back to the bridge, Phillips played for time until the Leader's impatience got the better of him.
Minus his weapon to prevent anyone seizing it, the Leader headed down into the ship's bowels to locate the crew himself. As the Leader rounded a corner, Mike Perry, the chief engineer, grabbed him and put a knife to his throat, bundling him into a secure room where the rest of the crew were hiding. Not long after, an unexpected message boomed out over the ship's loudspeakers. "Attention pirates. We have your buddy. We will exchange him for the captain." Panicking, the remaining pirates agreed to a trade, sealed by US$30,000 from the ship's safe that Phillips offered them.
As their own skiff had been wrecked during the hijack, the deal was that they would depart in the ship's motorised lifeboat.
But the moment the Leader was lowered into the lifeboat, they refused to let Phillips go.
The pirates headed off towards the Somali mainland with Phillips. But so did two US warships, which blockaded the lifeboat and ordered them to hand Phillips over.
A three-day pressure-cooker siege ensued, with Phillips and the pirates stuck in hideously hot conditions in the lifeboat. One of the warships' commanders managed to lure the Leader on board with the promise of a ransom. But with him gone, the rest started to panic, beating Phillips constantly and threatening to kill him.
Later, he scribbled a message to his family, realising it might be his last chance to say goodbye to them.
However, after a long wait to get three clear shots at the pirates as the lifeboat bobbed around, a team of US Navy Seal snipers pulled their triggers. "All of a sudden, I was sprayed with debris," Phillips recalls. "At first I thought it was the pirates shooting each other. The next thing, a US Navy Seal slid down a rope and I could hear him asking me if it was safe."
Covered in his captors' blood, Phillips was transferred to the USS Bainbridge, and eventually back to the US. So too was the Leader - real name Abduwali Abdukhadir Muse - who was later sentenced to 33 years in Indiana's Terre Haute prison.
"The sentence was justified completely," Phillips says. "This was a guy who told me he had kidnapped and murdered another captain, and who told me that I was going to die in Somalia. He is just a thug who doesn't care about other people."
Today, piracy is on the wane, thanks largely to ships starting to carry armed guards. But not every hijack has a happy ending, or such a prompt one. The average hijack now lasts eight months. And to this day, around 100 sailors are still hostage. Those who are eventually released often suffer serious post-traumatic stress.
And what of Phillips? Did the hijacking change him? "Not really. People say 'your life must feel great now', but my life was great before," he says. "What the hijacking taught me was that nothing is lost until we choose to give up, and we can all do a lot more than we think."
When I spoke to Phillips, he was just back from sea again, and has even sailed near the Somali coast. The ship he was on was armed, but a few suspicious-looking skiffs did sniff around occasionally.
Was he not tempted to stay at home in Vermont? "No, I still like being at sea, it's what I've done for 34 years and anyway, the wife is happy to see me back to work," he laughs. And so, the phone calls home continue. And, presumably, the Barry White impressions.