Faf du Plessis' 93-ball 100 underpinned South Africa's 325-6 and two pieces of magic from wicketkeeper Quinton de Kock — a cheeky run-out and a leaping, one-handed catch — left the Australians struggling on 119-4, with Khawaja also retired hurt with a left hamstring injury.
David Warner (122) and Alex Carey (85 off 69 balls) put on 108 for the fifth wicket but neither could see Australia home.
Khawaja hauled himself to the middle to bat after the fall of the seventh wicket, with his team needing 51 runs off the final 28 balls, and struck two fours in nine balls before departing for 18.
Australia required 18 to win off the last over — or 17 for the tie that would also have seen the team finish first on net run-rate — but Nathan Lyon was caught in the deep off the next-to-last ball.
Finch was adamant Khawaja didn't aggravate his injury by coming back out to the middle but said the No 3 batsman was unlikely to play the semifinal.
"He wanted to go out, especially when we got as close as we did," Finch said.
All-rounder Marcus Stoinis aggravated his side injury and also faces a scan.
The Australians might have fallen short but at least they have Warner in sublime form for the semifinals.
The left-handed opener has scored 638 runs in nine innings — just short of the tournament-high 647 racked up by India's Rohit Sharma — and now has three centuries to his name this tournament.
"That was as good an innings as he could have played," Finch said. "He is a big-game player, he loves the big stage."
Australia ideally needed him to last until the end of the chase but he succumbed in the 40th over to a superb diving catch by Chris Morris and departed to jeers, once again. Indeed, South Africa's catching was excellent in general, not least from De Kock who soared into the air to take Glenn Maxwell (12) soon after flicking a ball on to the stumps — while looking the other way — from Kagiso Rabada's throw to run out Stoinis (22).
Earlier, the South Africans got off to a fast start by reaching 73-0 by the end of the opening powerplay.
Lyon's offspin removed openers Aiden Markram (34) and De Kock (52), only for Du Plessis and Rassie van der Dussen (95) to quickly restore Proteas supremacy with a third-wicket stand of 151 runs.
Du Plessis reached three figures in 93 balls — and was dismissed off his very next delivery for 100.
- AP