In a test they couldn't afford to lose, Australia were slipping closer to an unwinnable position. The ball was 22 overs old. The initial opening few overs where the red cherry was swinging, and saw the downfall of first innings century maker Dean Elgar, were now over.
Australia needed wickets fast. Vice-captain David Warner had a plan to do just that - and he was going to get test novice Bancroft to do the dirty work.
It was a plan that has now ruined the careers of two of Australia's most valuable test players and one of their most promising young stars.
According to Cricket Australia's findings announced on Thursday - when the governing body revealed that Smith and Warner were handed 12-month bans and Bancroft a nine-month ban - Warner developed the plan to alter the condition of the ball.
The 72-test veteran approached Bancroft, who made his test debut in November, to take sandpaper on to the field in the second session and rub the rough side of the ball.
The desired result was to produce enough friction in a bowler's delivery that the ball would begin to reverse swing - an imposing prospect for South African batsmen facing the Australian pace trio of Mitchell Starc, Josh Hazelwood and Pat Cummins.
Warner demonstrated to Bancroft how to do it, while Smith was made aware of the plan and supported the illegal act. It was brazen, it was arrogant and it didn't go to plan.
Television cameras, apparently alerted by former South African cricketer Fanie de Villiers, captured Bancroft taking something yellow out of his pocket and rubbing the non-shiny side of the ball.
Host broadcaster Supersport aired the footage and commentators started to point out a possible indiscretion.
It wasn't long before Australian coach Darren Lehmann and umpires Nigel Llong and Richard Illingworth noticed the big screen and were aware of what was going on.
Cameras then caught Lehmann on a walkie-talkie communicating with Australian 12th man Peter Hanscomb, who ran on the field at the end of an over to talk to Bancroft.
According to Cricket Australia, Lehmann, who announced yesterday that he will stand down as coach following the final test which began in Johannesburg overnight, knew nothing of the ball tampering plan. "What the **** is going on?" was the message he sent to Hanscomb.
After words with Hanscomb, Bancroft was again captured by the cameras putting the yellow object down his pants before Illingworth and Llong approached. When they asked what was in his pocket, he produced a black cloth used to shine his sunglasses.
Play continued, the ball wasn't changed and some wickets fell but not at the frequency Australia would have hoped as South Africa got to stumps in a commanding position - a lead of 296 with five wickets in hand.
Yet the whole world by now had seen the damning and hard-to-explain vision.
At the end of play, Smith and Bancroft fronted a press conference to own up to ball tampering.
Bancroft said he used yellow tape, later confirmed by Cricket Australia as sandpaper, and Smith revealed the illegal act was decided by the "leadership group" - a story he has since changed.
"This is something I am not proud of. It's something I hope I can learn from and come back from. I am embarrassed. It is a big error in judgement," Smith said.
He no doubt thought the International Cricket Council would likely slap him with a ban and matters would rest there. Why not? That's been the case with every other ball-tampering case in the sport.
But this was different. This was Australia. What they had done was "un-Australian" and the Aussie press and the rest of the cricket media wouldn't let them off as easily. It was the underarm all over again.
While Smith, Warner and Bancroft were asleep in their Cape Town hotel, a level of hysteria took off in Australia as the new day there began. There were calls for life bans thrown about, and even Australian Prime Minister Malcolm Turnbull weighed in on the affair, demanding Cricket Australia act decisively in investigating the incident.
The next day, the reaction looked to hit the Aussies hard. Smith was dropped as captain and Tim Paine replaced him for the fourth day. On the field, Australia were set 430 to win and were bowled out for 107 in quick time.
An investigation was swiftly organised and sanctions handed out four days after the incident. Cricket Australia came down hard, sending the trio home. For being behind the plan and letting it happen, Warner and Smith were given 12-month bans and can play only club cricket over that time. Warner will never be able to captain the side again and Smith not for another two years.
The Indian Premier League also banned the duo from next month's tournament. With those lucrative deals, Cricket Australia contracts and sponsorship - the whole ordeal is set to cost Smith and Warner upwards of $5 million each, according to some reports.
Bancroft was handed a nine-month ban and like Smith and Warner, will have to live with the title of cheat for the rest of his career, whether or not it includes another test match.
Smith showed genuine remorse as he fronted a press conference at Sydney Airport on Thursday night, saying he would regret what happened for the rest of his life.
"If any good can come from this, if it can be a lesson to others, then I hope I can be a force for change," a teary Smith stated.
Change indeed. Things have already changed so quickly for the golden boys of cricket, two players seen as heroes in their country.
Cricket has seen quicker collapses, one at Eden Park courtesy of the English last week, but probably nothing more significant than the downfall of Smith and Warner, who have gone from cricket's biggest earners to club players.
• Smith and Bancroft have already fronted the media on their return to Australia. This morning, it's Warner's turn.
