So he asked the Navy's Deputy Chief of Operations Ronald Thunman.
"He said, 'All my life I've wanted to go find the Titanic,'" Thunman said.
"And I was taken aback by that.
"I said, 'Come on, this is a serious, top secret operation. Find the Titanic? That's crazy!'"
Thunman agreed to fund the Titanic expedition on one condition — that Mr Ballard use the money and the time to also locate two nuclear submarines that went missing in the Atlantic in the 1960s.
"It was very top secret," Ballard recalled.
The USS Thresher sank in April 1963 during deep-diving tests more than 300 kilometres off the coast of Boston. All 129 crew died.
Five years later, 99 crewmen died when their submarine, the USS Scorpion, mysteriously disappeared near the Azores, a Portuguese archipelago.
"So, it was a deal — you'll let me do what I want to do, if I do what you want to do," Ballard told CBS.
Ballard also spoke of the sombre moment he and his team found the ocean liner's wreckage.
"We realised we were dancing on someone's grave, and we were embarrassed," Ballard said.
"The mood, it was like someone took a wall switch and went click.
"And we became sober, calm, respectful, and we made a promise to never take anything from that ship, and to treat it with great respect."
The Titanic sank to the bottom of the North Atlantic Ocean after hitting an iceberg on April 15, 1912. Of the 2200 on board, 1500 people died, including John Jacob Astor, who was one of the richest men in America at the time.