Both Hammond, 45, and 52-year-old May have been supportive of Clarkson throughout, with the presenters insisting they would not work on the programme without Jeremy.
James ruled out claims that they could film the programme with a stand-in for Clarkson.
He said: "Me and Hammond with a surrogate Jeremy is a non-starter, it just wouldn't work. That would be lame, or 'awks' as young people say.
"It has to be the three of us. You can't just put a surrogate Jeremy in and expect it to carry on. It would be forced. I don't believe they would be stupid enough to try that."
He added: "It doesn't mean I won't go back - we may all go back in the future. It might just be we have a break from it. I don't know. It would be a bloody tough call to do Top Gear without Jeremy."
Hammond also appeared to dispel any such move, tweeting: "Not about to quit my mates."
He told reporters at a charity bike rally recently that the future will be "exciting" for the three men, but refused to speculate further on the future of Top Gear.
He said: "Everybody knows what's happened. That's all been in the news. The future, nobody knows yet do they?
"And I cannot and won't speculate further than that. So don't know, let's see. It'll be exciting whatever."
The BBC issued a statement recently, revealing that Top Gear Live would go ahead despite Jeremy's sacking, but after undergoing a rebrand.
It read: "So as not to disappoint the thousands of people around the world who have already purchased tickets, BBC Worldwide has agreed with our joint venture partner Brand Events that the remainder of the tour can continue. These events will not however feature any BBC Top Gear branding or content. We believe this is a sensible approach in the circumstances."
Creative Director Rowland French said: "The three presenters have been working incredibly hard to make sure it's better than ever before, and I'm certainly looking forward to seeing them back doing what they do best. If you love the three boys, then this is the only place to see them!"
Brand Events added: "This is a great solution for the fans. We'd like to thank our ticket holders for their continued patience. The fans are the most important people to Jeremy, Richard and James so we're delighted to be able to say 'we're still coming'. We're sure it'll be something you won't want to miss."
They gathered together in the market town of Petersfield in Hampshire on Thursday to prepare for the tour, after enjoying a meal at the Earl Of March pub in Sussex the night before.
The pub is just a 24 minute drive to Petersfield and about five minutes from Goodwood where the trio were said to be secretly filming after Chris Evans first sparked a rumour on on Radio 2.
The source said staff had no idea the Top Gear crew were the table of eight that had booked but Clarkson and his friend arrived first, followed by the others.
Everyone in the pub was quite relaxed, with no one asking for autographs although the group were placed in the private dining room, not in the main part of the pub or restaurant.
- Daily Mail
