"The more people told me I was a lunatic, the stronger my resolve grew to prove them wrong. In the end, when everybody was telling me I was a lunatic, it was easy. That made my mind up completely. I was like, 'You already think I'm nuts, so there's nothing to lose from here'.
"What's hilarious is when it all works out the same people turn around and go, 'Well you had to go through that to get where you are'."
Today, he shrugs off the cost of his defiance as "nothing".
He's the lead in an Emmy-nominated television series, makes movies alongside Jean-Claude Van Damme and Lindsay Lohan and returns Downunder to host The Amazing Race.
But it wasn't always so easy for Bowler to see that the end would justify the means.
Having carved out a television career on shows such as Blue Heelers and All Saints in Australia, he decided to have a go at cracking Hollywood with Australian actress and then-wife Roxane Wilson.
She became pregnant with daughter Edie shortly after their arrival, making the financial struggles of the following years even more harrowing as the couple tried, unsuccessfully, to make their name in a town already flooded with desperate actors.
Welcoming their second child, Zeke, in 2005, Bowler at one point took a job cleaning elevators.
"The first four years were crushing. I was coming and going, I couldn't get a job, all these 50/50 calls went the wrong way.
"The lowest point was when Roxane and I were together. We had sold our home in Australia, had come here again two more times and were broke, with two kids. I felt like the world's worst father because I was dragging my kids around stupidly.
Grant Bowler became a recognised face on New Zealand TV screens with his role as Wolf in Outrageous Fortune.
"There was always the reality that countered that though. For male actors in New Zealand and Australia over the age of 40, the jobs get very thin on the ground. It's very difficult to sustain an income and it's very difficult to give your kids the future you want to give them. So if I was going to stay in the industry and give my kids the future I wanted to, we had to go through that.
"I remember I did an interview once about how rough things were and people commented saying, 'How irresponsible' and 'If you really cared about your kids you'd stay put and just be grateful for what you've got.' I read it and was gutted. I was heartbroken because at that point I didn't know whether everything would work out.
"But the one thing I don't ever want to teach my kids is, 'Be grateful for what you've got.' It's a classic Kiwi and Australian line. 'Shut up and don't ask for anything more.' I don't want them learning that. It just means, 'I'm shit-scared of failing.' The most important lesson I've ever learned is to have a healthy relationship with failure."
At times, Bowler's fearlessness wasn't enough and potentially life-changing opportunities crumbled amid bad timing.
Like when he auditioned for one of his favourite shows, Big Love, starring one of his favourite actors, Bill Paxton.
It was the 11th hour of yet another trip to Los Angeles - his last chance at an acting breakthrough before having to fly home to Australia, lacking enough money to buy even one more meal.
The role went to someone else but Bowler's saving grace came in the form of a call from Outrageous Fortune.
In a "heartbreaking" twist, immediately after hitting the Outrageous set in Auckland, the call came that Big Love had loved Bowler so much that they had written him a new role.
"These are HBO writers for Big Love ... that's a massive show! If they were willing to write me a role I knew I would work in LA eventually. But I couldn't take the role, I was broke again, I had this inside knowledge that one day it was going to be okay, but everyone around me was still going, 'You've got to stop doing this'."
But even after Outrageous, Bowler could not give up on his Hollywood dreams. "Well, I could have. But I didn't. In the end I came to the decision that if I went completely bust and was broke forever, it didn't matter. Plenty of people live like that. Plenty of people don't have a lot of means. Plenty of people choose to be artists, even when they don't make a good income. So what?
"You get one go-around, so you have to go after what you what. I had a sneaking suspicion that if I stopped trying I was settling. I didn't want to settle."
His perseverance finally started to pay off with roles on Ugly Betty, Lost and True Blood, alongside Anna Paquin. Films including The Killer Elite opposite Robert De Niro and Atlas Shrugged also followed.
But as his US success kicked into gear, his marriage was falling apart.
Grant Bowler in Ugly Betty.
The couple divorced in 2011, Wilson telling a reporter that the split was "one of the costs of the journey". The Home and Away actress remained in Australia with the couple's two children, while Bowler continued to commute between the US and Sydney, at one point flying back to Australia 13 times and once again grappling with the guilt of feeling like the world's worst dad each visit - blatantly unaware how little his lack of money affected Edie and Zeke.
"I always felt terribly guilty because I would fly to Sydney, rent a car, pick up the kids, and then we'd go to this little motel because I didn't have any money. For years, I felt guilty about that.
"Last year my kids and I were lying in my now king-size bed reminiscing, and they said, 'Daddy, do you know what we miss the most out of everything? That motel. Because you would always be so tired that you'd fall asleep in bed with us.' It was one room, one bed and no toys, so we had to play with each other all day then we would sleep in the same bed, snuggled up together night after night.
"Those years that I felt really guilty about it being just us in a motel room are my kids' fondest memories. That's hilarious to me."
Wilson eventually relocated back to Los Angeles and the pair now live 6km apart, sharing parental responsibilities. After our interview, Bowler was driving Zeke to his banjo lesson, while Edie takes piano and cello classes.
The children came to terms with their parents' divorce "a long time ago", says Bowler. "Even though the marriage didn't work out, we both want to parent the kids together. That's really important to us and super-important to the kids. I run around with work a bit, but it works out pretty well."
Now dating architect Kate Buchwald, Bowler is not convinced that Los Angeles is the greatest place to raise children and admits being particularly fearful about having his daughter grow up in an image-obsessed town. He's yet to deal with her first boyfriend, but says the first crushes have begun.
"I think girls can find it particularly tough in LA because it's the mecca of what's considered beauty, sexiness and good looks. I don't want her getting caught up in that. The culture here, more so than at home, is about being aspirational and with girls, they get sexualised young.
"For boys, it's the issues that are shared back home, like teen depression. With teenage boys it's like you've just got to keep them alive until they turn 18 and, unfortunately, in a city like this there are a lot more chances for them to get into a lot more trouble so yeah, I do worry ... At the same time I think we're doing a pretty good job."
Meanwhile, Bowler's career has hit an all-time high since he landed the role of Joshua Nolan on Defiance. The series, which starts screening on Sky on Monday, is accompanied by a parallel video game, which Bowler also voices - and plays after the kids have gone to bed, in a bid to "keep them off Xbox as long as possible".
Director Kevin Murphy says Bowler quickly established himself as a natural leader on the series.
"There's a certain art to being number one on the call sheet," says Murphy. "A leader in that position sets the tone for new actors, guest stars and directors, and for the attitude and morale of the crew. Grant is such a wonderful dad to everyone. He's loving and joking, but you also don't want to cross him.
"You don't want to come to set not knowing your lines or not having carefully studied your alien languages, because he'll give you that look and it will make you feel like you're 4 years old.
"This [cast] genuinely love being with each other and that starts at the top, with Grant. And he's a great actor!"
For Bowler, after 20 years in film, television and theatre, he had "said every line you can say in one form or another" and was drawn to a role that allowed him to explore behaviour and unleash his inner action hero.
A scene on US prime time series Lost featuring Grant Bowler.
"I got to 40 and realised I've got to do all the running and action stuff I possibly can because there will come a day, maybe 55, maybe 65, where it's not for me anymore.
"It's like when I was doing True Blood and was naked all the time. While someone still wants to pay money to see me naked I'll do it, because there will come a day when people say, 'Put your gear back on!'"
Filming in Toronto ("I challenge anybody down in Invercargill to match a Toronto winter," he says), the show has seen him garner an epic following, which he cultivates by attending frequent fan conventions.
At times, he still gazes into the crowds in disbelief that fans are dressing up as his character.
Looking back at the Hollywood ambitions he was chasing when he first arrived in the US, there's no doubt Bowler is sitting atop his dream.
"This is it," he smiles. "Which is really funny because I now have a completely different set of goal posts.
"Right now I have what I always wanted - lead on a cable show, 13 episodes a year, I get to do action stuff, the show's successful, I'm happy and I can make indies in my off-season. It's perfect. There's nothing I would change about it right now. But I've also come up with a whole bunch of new stuff I want.
"I'd like to start putting projects together and I might like to direct one day soon ... which I've never said out loud before but I think it's about time. More and more my producers and directors have asked me when I'm going to start producing and directing. On Defiance my crew ask me every week.
"When other people are making that suggestion constantly, it's probably time to look at it. With acting you're single-focused and all you worry about is what you're doing, whereas with these other jobs you're worrying about what everyone is doing and everything is out of your control, which is very challenging.
"So there are definitely new things I want to explore. I might be terrible at all of them, but it doesn't matter. I do not, however, want a rap or singing career or to release a line of perfume."
He'll leave all that to the likes of Lindsay Lohan, one of his renowned co-stars - the pair portrayed Elizabeth Taylor and Richard Burton in the 2012 television biopic, Liz & Dick.
"I walked in one morning and we were all exhausted, staring at each other wondering what was going on and one of the crew looked at me, smiled and said, 'If you buy tickets to the circus, you'll probably see clowns.' "That was Liz and Dick." He first met Lohan at the chemistry read in LA and says she was "fantastic".
"We had a great time. She's very, very intelligent. It sounds terrible, but I'm always surprised when somebody who has that many paparazzi chasing them is very, very smart."
So what goes through one's mind when filming sex scenes with one of Hollywood's most scrutinised celebrities?
"The same as what goes through your head when you're shooting intimate scenes with everybody else ... which is, 'Is my junk showing?'"
Bowler takes on the role of town sheriff in new sci-fi series Defiance.
With several more movies due out in coming months, like Swelter with Jean Claude Van Damme, and two children's films including Zoey To The Max "because my kids were complaining they can't watch anything I make", Bowler can for now relish his long-fought-for job security.
He's just turned 46 - "I never hide my age, I've earned every minute of it" - and spends his downtime with the kids, exploring LA's culinary scene, sifting through book stores and attending Tinseltown events with his girlfriend.
And, though he doesn't often run into the huge community of Kiwi and Australian expats, he remains close to Anna Paquin and husband Stephen Moyer, who he worked with on True Blood.
"Steve ended up becoming my bestie. Anna's one of the world's great people and the two of them saved my butt when I was here on my own. Socially they helped me, then when I was broke they moved me in next door to them. They gave me a place to live - I only just moved out in October. I've got nothing but time for 'Paccas'. We're great mates."
LA is home "because that's where the kids are" but he admits missing the people, intimacy and culture of New Zealand, adding, "someone needs to give me a bloody job in New Zealand so I can come back".
But no matter how long he stays away, Kiwi qualities continue to tide Bowler over.
"I identify as an 'Aussiwi' because I was raised by both. A mum, who was quintessentially Kiwi and a dad, who was quintessentially Australian. I never realised how influential that was until I went back to do Outrageous and people would say things or behave in certain ways and I would think, 'Oh, that's Mum.'
"I finally got to the age where I could differentiate between what the person was and what the culture was and I realised that my mum reacts in certain ways because culturally she's a Kiwi and my dad reacts differently because he's an Aussie.
"The great thing is that I have both in me - and that defiance is definitely the Kiwi side. It's my mum all over!"
Defiance starts on November 3 at 8.30pm on new Sky channel, The Zone.
