Those days may be gone - though it was only 1970 when slavery was formally abolished in Oman - but the seafaring tradition is alive and well in Sur.
When I climbed the small headland on which the lighthouse stands, I could see the harbour was full of craft - ranging from small rusty trading ships and modern yachts to the fibreglass speedboats used for fishing everywhere along the Omani coast, and a few magnificent wooden dhows like the one resting on the foreshore.
In the town centre a fish market, made of concrete but built in traditional style with open sides, allows women in their black hijab robes to haggle directly with turbaned fishermen for freshly landed fish, as they have for centuries.
In the narrow lanes near the harbour you can walk past mudbrick buildings with high walls and large ornate gates, once the homes and warehouses for global traders, and imagine lines of slaves carrying in bundles of spices from India, silks from China and animal skins from Africa, as they did not so long ago.
The town continues to be dominated by the 300-year-old Sunaysilah Castle, a powerful square of battlemented walls with four round towers standing on a strategic knoll, its canon aimed equally at the sea before, the desert behind and the town below.
The castle was restored recently with the aid of Unesco funds, its rebuilt rooms lined with old guns, daggers and powder horns, its outbuildings sign-posted as "prison", "water cistern" and "Koran School".
But Ali, our guide, who was of African descent, whispered quietly that most of these places had actually been used to hold slaves.
"No one wants to talk about it. They pretend it didn't happen."
There is, however, continuing pride in the tradition of dhow-building. Down on the seafront is a new museum - so new it hadn't opened when I visited - celebrating these remarkable boats.
Taking pride of place in the courtyard outside is a 150-year-old dhow, built in Sur, traded around the world and brought back from its final resting place in neighbouring Yemen for the museum.
Would it have carried slaves, I wondered?
"Oh, yes," said Ali.
"Slaves were still being brought here and to Yemen more recently than 100 years ago. This ship would have carried them."
The most impressive tribute to Sur's past, however, is not the museum but a working dhow factory which is still churning out these lovely old ships.
Inside its rough wooden gate I was confronted by a massive and seemingly chaotic pile of timber assembled from around the world, and a jumble of old sheds.
Inside one of these sheds we found two craftsmen building a model dhow, about three metres long, evidently in demand as a cultural touch to five-star hotels, posh offices and luxury homes.
Inside another shed was a small shop where miniature dhows of varying sizes were for sale.
Further in, lined up along the shore of Sur's lagoon, three full-sized vessels were being constructed under rough shelters.
First, a magnificent old dhow was being modernised for a wealthy Arab.
"They are very popular as pleasure boats," explained Ali.
"The Sultan has one ... though he also has a modern ship.
"But these boats are not like the old ones. They have air-conditioning, diesel engines, refrigerators, gold bathrooms - anything you want."
Next in line was the keel of a new dhow that had just started to be laid, the curved timbers already pointing to the elegant shape of the finished craft, but no one was actually working on it when we were there.
Down the end, three craftsmen were placing the final timbers, planks about 5cm thick, along the top of a nearly completed hull, and arguing furiously about the best way to do it.
"It is not just Arabs buying these," said Ali.
"Dhows are traded all round the world, and they are part of the history of many countries."
That's true. I've read that Chinese records tell of an Omani dhow visiting Canton about 1300 years ago. It's rather nice to know that these ancient craft still have a part to play.
But the other end of the shipyard tells a different story. On the foreshore lies a dilapidated old dhow which looks as though it is slowly falling to pieces. Beside it in a long shed with open sides sits a row of the ubiquitous fibreglass fishing boats.
Fibre glass? In this temple to wood and craftsmanship?
"It is cheaper and easier to build with, and there is less maintenance," said Ali.
"That is what most people want."
CHECKLIST
Getting there: Etihad Airways and Air New Zealand operate a codeshare partnership from Auckland, Wellington and Christchurch, via Australia, to Abu Dhabi. Transtasman flights are with Air NZ. Etihad operates 11 flights a week from Sydney and daily flights from Melbourne non-stop to Abu Dhabi. From its base in Abu Dhabi, Etihad flies to to 86 destinations around the world including Oman.
What to do: World Expeditions operates nine- and 12-day adventures in Oman.
Further information: See walshegroup.com.
Jim Eagles visited Oman with help from Etihad Airways, Air New Zealand, World Expeditions and Oman Tourism.
