Fair to say Rathman had his doubts about Hayne's chances of ever becoming an NFL player when he arrived at Levi's Stadium in April, 2015.
"It was funny speaking to my running back coach when I left," Hayne told former NRL star Denan Kemp's The Locker Room podcast.
"When I told him I was playing rugby, going back for the Sevens. He was like 'man, it's been a great journey, man. I really loved what you did'.
"I said 'what were you thinking when I first came here?'
"He goes 'man, every day I'd go up to the planning committee and say "you guys are idiots. What is this kid doing? He cannot play. He has got no idea what he's doing'".
"Obviously Tom was an awesome guy. He said it wasn't until I put pads on that he really had faith in me and would give me a crack. First week I went from the third string running back to the second string running back. By week 15 I was the starting running back. It was a huge learning curve."
Hayne said it was only a fateful USA Visa technicality issue that prevented him from signing with the Seattle Seahawks in 2015.
He couldn't be happier that his negotiations with San Francisco worked out.
"The 49ers were awesome," he said.
"It was such a great environment for me to go because they had a coach who had experience with foreign players. I had a special teams coach that really understood where I was coming from. Like all Americans, you really have to prove yourself. It took me a long time to really prove myself.'
In May, the 28-year-old spectacularly left the San Francisco 49ers to play Rugby Sevens in a bid to win an Olympic berth with the Fijian team.
While some thought he may have been tapped on the shoulder by new 49ers coach Chip Kelly and told he wasn't going to make the final cut, Hayne has put this theory to bed, saying his reasoning revolved around game time.
"I need to be getting game time and you just can't get that over there (in the NFL) because of the way it works," Hayne said.
"The offence is only on for a certain amount of time and then there's five other running backs.
"If there was a second division team where I could get those mental reps of being on the field, 100 per cent I'd go back to the NFL.
"I'm at the end of my career where I really (have) just got to get on the field. I just want to be on the field. I've played eight games in two years.
"If I went back there to the NFL it'd be one of those things where I'd be a second or third string guy helping out but not really getting a lot of game time."
