The actor, who's also known for Lord of the Rings and Shogun, found it read like a comic and told his agent that although it was "wonderful stuff", it would either be the biggest catastrophe of all time, or set a new fashion in filmmaking.
Four movies later, as well as one of the most iconic film characters and a reported combined box office of about $939 million, makes it pretty clear which of Rhys-Davis' instincts were correct.
Now, Indy is getting a modern makeover, with Raiders of the Lost Ark (1981) lovingly restored and remastered with painstaking detail by Spielberg and sound designer Ben Burrt.
Indiana Jones and the Temple of Doom (1984) and Indiana Jones and the Last Crusade (1989) have been remastered and all three are being released in a Blu-ray boxset with Indiana Jones and the Kingdom of the Crystal Skull (2008).
The boxset includes seven hours of bonus materials, including something of a "Holy Grail" itself for fans - a brand new two-part documentary from the set of Raiders of the Lost Ark.
"There was a sort of freshness and enthusiasm about it, that made the first one hard work but absolutely delightful," Rhys-Davies remembers about Raiders of the Lost Ark.
It was also hard work, because apart from the desert heat of Tunisia, he and most of the cast and crew also caught a bug, during which he says he lost 10kg in two days.
Spielberg was one of the only ones to avoid getting sick, because he was sneaking tinned food on the side.
Rhys-Davies says he wanted to be a part of Indiana Jones to work with Spielberg and compares the experience to seeing a young Mozart at work - an "effortless, bubbling creativity".
"But what's interesting, and seeing more and more of the behind-the-action stuff, particularly on this Blu-ray, you realise more and more how the film is the product of these three geniuses," he says, referring to Spielberg, screenwriter/producer George Lucas and actor Harrison Ford.
"And Harrison's role is not to be underplayed," Rhys-Davies says.
"What the behind-the-scenes stuff gives you is the sense of an actor who is very deliberately and very firmly defining an iconic character in a way that I don't think I've ever seen displayed by any actor before...
"I think it's a tour de force."
Rhys-Davies says he still catches up with Ford, joking it's "generally in unusual circumstances that can't be talked about".
But he says he respect for Ford continues to grow as the years pass.
"I think he was an ambitious young actor out to make a name for himself from Han Solo (Star Wars) days.
"And he's moulded himself and grown and matured as an actor and as a star, that really you can make the argument, that he's the most significant old fashioned, old-time film star of his generation.
"And that's a remarkable achievement."
* Indiana Jones The Complete Adventures are available on Blu-ray from September 12. Win a copy here.
- AAP