They had the Wallabies on the ropes and were unable to land the knockout blow, but Gatland backed the decision of his captain Sam Warburton to push for a try instead of kickable penalties.
"We've got to give credit to Australia...that defence was heroic and I'm sure they're proud of that performance.
"But the guys know how close that was... we got over the line three times," said Gatland.
"It was a tight, tough test match and I don't think there was much in it."
His side now face a rejuvenated South Africa, fresh from a 64-0 win over the United States, and a team which has sparked into life after the shock defeat to Japan.
However, Gatland did not believe the physical and emotional toll of the tough Pool of Death matches would be too hard to overcome.
"The players have already talked about [the quarter-final] in the changing rooms. Our mind has already turned to next week."
Read more: Courageous Australia confident of chances
And Gatland said Wales had the confidence they could beat the Boks.
"We did it 12 months ago....we've had a couple of losses too, including one where we were leading by 17 points. And we've learned a huge amount from that."
A wry smile crept onto Warburton's face when asked about his decision to kick for the corner, when trailing 6-12, instead of a penalty.
English captain Chris Robshaw made the same decision a few weeks back against Wales and it led to defeat - and ultimately an early exit from the tournament.
"I felt we needed a try to beat Australia," said Warburton, who pointed out Wales needed two penalties to draw even.
"I thought we had to back ourselves, when they were one or two men down...we have to give them a lot of credit to defend that.
"It's a big lesson; you spend that much time in their 22 you've got to come away with a try."
Despite the deflating loss, Warburton was pleased to stop Australia - "one of the best attacking teams in the world" - from scoring a try and believed his side was mentally prepared for the knock-out match against the Springboks.
"That's what great teams are made of, being able to bounce back. It was disappointing to lose today but we back ourselves to come back next week."