But after Lemoine's red card in the 58th minute, Cavani turned in Maxwell's cross from the left 10 minutes later for his seventh goal of the season and then Matuidi's lofted pass beat goalkeeper Stephane Ruffier after bouncing in front of him with Cavani almost getting the final touch.
"I am proud of my players, but I feel disappointed for them because we gave everything and we caved in with a player less," Saint-Etienne coach Christophe Galtier said. "It's a shame, but the crowd can be proud of them."
Earlier, Monaco beat Lyon 2-1 thanks to Radamel Falcao's eighth goal of the season and Mounir Obbadi's first.
PSG spent most of the first half on the back foot as Corgnet went close with a shot from 25 meters and then scored in the 18th minute following a bad error from PSG center half Marquinhos.
The Brazilian was far too casual as he cleared the ball and hit it straight to Hamouma. He then teed up Corgnet just inside the penalty area and his first-time shot flew past PSG goalkeeper Salvatore Sirigu and into the top right corner.
Brazilian striker Brandao went close to doubling Saint-Etienne's lead but dragged his shot just wide of the post in the 29th, and midfielder Loic Perrin headed wide from a corner for a dominant home side.
The second goal came in the 51st when Sirigu came out to claim a cross and fumbled the ball as center half Alex jumped up at the same time as him. The loose ball fell to Marquinhos, but his terrible clearance bounced off Thiago Motta's leg and fell to Hamouma, who clipped it into the net.
"If you don't play well against teams like this you're in trouble and that's what happened," Blanc said. "We made two huge errors in defense and we were punished."
PSG was thrown a lifeline when Lemoine got a second yellow card for a foul on Ezequiel Lavezzi.
Lemoine, with blood pouring from his face following the challenge, had to be restrained and then dragged away by Saint-Etienne's staff as he furiously contested the decision.
"We deserved the win, but we have to keep this attitude if we want to do well this season," Hamouma said. "The red card changed everything."
It became increasingly hard for Saint-Etienne to hold out and Cavani's goal set up a frantic finale.
"We showed that we have character and heart and we're still unbeaten," Matuidi said. "But we came up against a great Saint-Etienne side and a great crowd. We lost games like this last season but this time we came back to 2-2."
Earlier, Obbadi gave Monaco the lead with a powerful half-volley into the top right corner.
Falcao added a another goal in the first half with a clinical finish before striker Bafetimbi Gomis pulled a goal back for Lyon after the break.
Falcao had not scored in his previous three league games but put an end to his mini-slump in fine style.
A long pass from defense deflected off the back of midfielder Joao Moutinho and fell to Falcao. He sprinted into the penalty area down the right, beat the left back for pace and expertly chipped the ball over Lopes as he charged off his line.
"Falcao works hard for the team, he closes down, he holds the ball up," Monaco coach Claudio Ranieri said. "Everyone should try and play for him more."
Also, Cheick Dibate and Ludovic Obraniak scored late goals as Bordeaux beat 10-man Montpellier 2-0.
