When "the Pig" opened there was a brewery on the roof. With the heavy lifting involved it was not "the ideal setup" and it was sold shortly after he took over, Mr Brown said.
In the early days meals were served until 5pm with just a snack menu after that.
He changed that pretty quickly as well, although not everyone was a believer.
"The chef and I sat here on the first night we opened for dinner, he shook his head and said 'it's never going to work'."
The pub now sells about $1 million worth of food every year including its famous curly fries.
Before buying the pub, Mr Brown lived in Canada where curly fries and chicken wings were big. He tried them out during the early days.
"The curly fries stuck, the chicken wings didn't."
With a background in bungy and retail management, Mr Brown said he was "at a bit of a loose end" when the then owners suggested he buy them out.
The rest is history even if he did have a bit to learn about the hospitality industry: "I didn't even know how to pour a pint".
These days there's a general manager to take care of day-to-day operations while Mr Brown focuses on strategy and keeping the bar fresh.
"We've always been prepared to reinvest for the long term." There are plans to "spend a bit of money on the place" in 2016, including the garden bar.
Mr Brown said the smoking ban that came into force in 2004 was one of the best things to happen to the Pig and for that he thanks Rotorua Mayor Steve Chadwick, then Rotorua MP, who promoted the bill.
"We were very much for it. We felt like we couldn't be the first one to do it alone, we needed to keep the playing field level," he said. "For the first 12 months there was a bit of a downturn in trade but after that it was much better."
One of the toughest times was an issue with noise control more than 10 years ago.
The council removed a wall across the road which meant noise reached an apartment building - resulting in numerous noise complaints.
"The council threatened to shut down the live music ... in the end they recognised they contributed to the problem and we found a resolution to it."
Mr Brown said there's been a real change in drinking culture over the past 20 years.
"The good old days of five o'clock drinks and stay till midnight - there's a lot less of that. People now stay for one or two, maybe something to eat, then go home." A greater awareness of drink-driving and the social cost of binge drinking contributed to that as did as a raft of legislation - not that that affected the Pig much, he said.
"We've always been more than compliant, always expect good behaviour.
"Millions have been through over 20 years, we've had very few problems really in the scheme of things."
Mr Brown said he's had great staff over the years - the pub now employs about 30 people. And although marketed to tourists, it's also stayed a firm favourite with locals.
Mr Brown, who also started Capers cafe 11 years ago, said he's pretty well entrenched in his pub.
"I don't know why I'd sell it," he said.
"I like to think it's part of the fabric of Rotorua now ... as much a part of Rotorua as some of the tourist attractions."
