But Wales, which was down to 12 men at one point with three players in the sin-bin in the final minutes, somehow found enough grit to score from its own rolling maul and force South Africa to scramble desperately for a victory.
Former captain Alun Wyn Jones came back from the bin just before the try, but Wales still had only 13 players on the field when replacement forward Dewi Lake drove over in the 77th minute to tie the score at 29-29.
Wales captain Dan Biggar shaved the outside of the right post with his conversion attempt and gave away the critical penalty — an intentional knock down of a pass — to allow Willemse to save the Springboks in the third minute after the fulltime hooter.
"It was a shame they got away with it at the end, because I would have taken a draw," Wales coach Wayne Pivac said.
"It was a big step up form our last performance but the discipline is clearly disappointing. I'm very proud but very disappointed, it is one that got away."
Wales was written off completely before the test, because of its record in South Africa and a poor Six Nations campaign that included a shock loss to tournament minnow Italy.
But Wales had the top-ranked Springboks' number in the first half.
Rees-Zammit scored the first try in the third minute as Wales stole a South Africa lineout and went from the far left of the field to the far right to create an overlap for Rees-Zammit to run in unchallenged.
The winger had his second off the back of a lineout that went wrong for Wales, but where he reacted quickest to kick the ball through, collect and touch down amid weak South Africa defense.
The Springboks were playing in front of spectators at home for the first time in three years because of coronavirus restrictions. The Welsh silenced them early.
"I'm gutted. There was a lot of momentum shifts in that game and we thought we nicked it but it wasn't to be," veteran Dan Lydiate said after his first test in 17 months.
"We were disappointed after the Six Nations and we wanted to put in a good performance today to restore pride in the jersey, but we came here to win and we're gutted with the result. We have to dust ourselves off and go harder next week."
Despite being thwarted regularly by Wales in the first half, the Springboks kept faith with their No. 1 weapon, the rolling maul, and hooker Bongi Mbonambi and his replacement, Malcolm Marx, both had tries from mauls in the first 11 minutes of the second half to close to 18-15.
Biggar's boot pushed Wales 24-15 ahead but South Africa kept coming, and center Damian de Allende threaded a low kick through for wing Cheslin Kolbe to score in the 65th. South Africa created mountains of pressure before finally getting its penalty try and the lead with five minutes to go.
Wales, stubborn and brave, wouldn't go away though and Lake's late try again forced the Boks to dig deep.
Willemse, who was selected at fullback, kept his nerve having stepped in at flyhalf and taken over the place-kicking duties when starting No. 10 Elton Jantjies was substituted at halftime.
