In a tough night for Newcastle, Test back-rower Beau Scott limped off with an ankle injury in the 56th minute and Tariq Sims was put on report for a high shot just 14 minutes into his debut for the club.
Meanwhile, Blake Austin starred for the Raiders in a man-of-the-match effort, alongside his new halves partner Sam Williams, as Canberra fought off a late Manly fightback to run out deserved five-tries-to-three winners.
Manly are 1-4, their worst start to a year since 2009 when they lost their first four games.
The perennial finalists are in a dogfight to qualify for their 11th successive playoff series. The Sea Eagles would have loved to have circled the wagons at Brookvale Oval, but took this match to Albury's Lavington Sports Ground.
Missing a host of stars including skipper Jamie Lyon, Steve Matai, Jorge Taufau and Brenton Lawrence to injury, Toovey said Manly were beaten by a more-dedicated team.
"It's a long season but we need to turn things around quick. I said that last week as well," Toovey said.
"It's a different challenge for all of us. "We have been here before and managed to dig ourselves out of it."
One of the few highlights for Manly was the performance of young gun Tom Trbojevic. The 18-year-old scored the first of two tries in his initial touch of the footy after just two minutes and put in a strong all-round performance.
Coach Ricky Stuart was full of praise for the way his Raiders side bounced back from their hammering at the hands of the Sydney Roosters last week, particularly the efforts of Austin and Williams.
"He put himself out there which is what I really like about Blake," Stuart said. "His defence was really good - we have been working hard on that.
"Everyone knows he is a good attacker but he aimed up tonight and was really good.
"Sammy was really important in the direction he gave the team. He put us on the front foot and put pressure on the opposition with his kicking game."
After Trbojevic's first four-pointer, Sisa Waqa opened the Raiders' account in the 11th minute before Austin stepped his way over just three minutes later.
Austin then put Jarrad Kennedy over in the 24th minute and the Raiders led 16-4, until Jordan Rapana crossed four minutes before the break after Waqa tapped down a Williams kick.
A Williams field goal in the shadows of halftime put the Raiders further ahead. Jarrod Croker crossed out wide in the 54th minute and converted to give his team a near unassailable lead of 29-4.
That was before Manly mounted a late comeback. Trbojevic crossed for his second after a lovely tap-on from Brett Stewart in the 60th minute. Cheyse Blair then latched onto a clever cross-kick from Kieran Foran nine minutes later to set up a chance of an unlikely Sea Eagles win, but Canberra held firm.
- AAP