It was all the home side in the opening exchanges, Sam McKendry barging over from close range, and then Bulldogs captain James Graham put on report for a shoulder charge on Campbell-Gillard.
Coach Anthony Griffin's decision to start Peter Wallace at hooker and Tyrone Peachey in the halves also seemed to pay off, with the Panthers rolling through the ruck with ease.
But the visitors hit back through a 24th-minute try to Will Hopoate to go into the sheds down 14-6.
Despite failing to find the tryline in the second half, Penrith - led by five-eighth Jamie Soward - still appeared destined for victory until Mbye scored to reduce the gap to four.
Tolman came off with a concussion test in the 17th minute and didn't return, while Penrith centre Dean Whare also succumbed to a calf injury late in the second half.
Bulldogs coach Des Hasler said his team was "particularly ugly", but praised Holland for his clutch game-winner.
"It's a debut that dreams are made of," he said.
"It'll be particularly memorable for him, this one. I thought he was steady tonight and came up with the big plays when he had to."
Graham said he was also proud of the rookie and was confident the kick would sail through.
"I had all the belief in the world in him to slot it over. I think it meant a lot to him up there. Very proud of him," he said.
Panthers coach Anthony Griffin was upbeat despite leading right up until the final siren, and was encouraged by the 80-minute effort.
"As a team, obviously it's heartbreaking tonight, we got beaten at the death. But as a team, we're getting a little bit better every week and learning the lessons that we need to learn," he said.
"I can't wrap us enough about our effort and the way we're going about things."
-AAP
