And his impact increased when his character Dr Derek Shepherd became leading lady Meredith Grey's love interest.
Parriott recalled creator Shonda Rhimes saying, "He's McDreamy. He's the perfect man", of Dempsey.
Meanwhile, writer/producer Jeannine Renshaw recalled, "We all love Patrick. Patrick is a sweetheart."
But others thought he held too much sway over the production.
"He had this hold on the set where he knew he could stop production and scare people," Parriott said.
"The network and studio came down and we had sessions with them. I think he was just done with the show. He didn't like the inconvenience of coming in every day and working. He and Shonda were at each other's throats."
That tension spilled over into his relationship with on-screen love interest Ellen Pompeo, who became "frustrated" with him and "would get angry that he wasn't working as much", according to Renshaw.
"She was very big on having things be fair. She just didn't like that Patrick would complain that "I'm here too late" or "I've been here too long" when she had twice as many scenes in the episode as he did,' she explained, later defending him by saying: 'He would get it. It's just that actors tend to see things from their own perspective."
The star himself recalled how gruelling the shooting schedule was.
"It's 10 months, 15 hours a day. You never know your schedule, so your kid asks you, 'What are you doing on Monday?' And you go, 'I don't know,' because I don't know my schedule. Doing that for eleven years is challenging," he admitted.
But as complaints from co-stars and crew stacked up and Dempsey's relationship with Rhimes went south, his days on the show were numbered.
Renshaw said, "I think Shonda finally witnessed it herself, and that was the final straw."
"Shonda had to say to the network, 'If he doesn't go, I go.' Nobody wanted him to leave, because he was the show. Him and Ellen," she continued, adding: "Patrick is a sweetheart. It messes you up, this business."