Rondo scored 11 points in the final quarter and notched his ninth career playoff "triple-double".
As the final seconds ticked off the clock, Boston fans began chanting "Beat the Heat!".
Celtics' veteran Kevin Garnett, who had 18 points and 13 rebounds, echoed the sentiment.
"Let's do it!" Garnett said of tackling the Miami team led by NBA Most Valuable Player LeBron James and Dwyane Wade.
It will mark the third straight post-season that the Heat and Celtics meet.
Boston won in five games in 2010 in the first round and the Heat won in five last year in the second round.
Brandon Bass scored 16 and Pierce ended with 15 points and nine rebounds for Boston.
Andre Iguodala scored 18 points for Philadelphia, which had eliminated injury-plagued top seed Chicago in the first round.
Iguodala drained a three-pointer from the left corner with just over four minutes remaining to pull Philadelphia within 71-68.
Pierce then fouled out, but Rondo made sure the Sixers couldn't take advantage.
-AAP