"Our lease was up so it was decided we would vacate," Ellis said.
The restaurant, which opened in June 2010, will close its doors permanently this Saturday.
The two bars run under the same licence in the same building, Ratbags on the bottom floor and Innocent Bystander on the second level.
Ellis and his two brothers, Barry and Grant, have a long history in the hospitality industry in Dunedin and Central Otago, with a combined 100 years' experience.
Phil Ellis said the restaurant was formed following a marketing trip to Melbourne, where the three brothers visited more than 50 bars in two days.
"There was a compulsory drink at each one," he said.
Following that trip, the Innocent Bystander opened with what was thought to be Dunedin's first "topless bar", where the deck outside did not have a roof so punters could sit looking out over the Octagon.
"It raised a few eyebrows but it was a bit of humour," he said.
Ellis told the Otago Daily Times yesterday they had "absolutely no regrets" about being in business.
"We loved it, every minute of it."
The losses associated with Covid-19, along with the lack of tourists, events cancelled, and the loss of cruise ships made the business a casualty of the pandemic.
"We can't survive in this environment and we don't see it changing in the short term," Ellis said.
While losing the business was tough, Ellis said there had been plenty of memories made along the way.
Having the 2011 Rugby World Cup village in the Octagon, sponsoring musical shows at the Regent Theatre and Otago Rugby, plus receiving countless compliments on the one-metre pizza and triple cooked chunky spuds, were just some.
However, the people who had come through the doors were the main highlights.
"From all walks of life from all differing races and cultures - all interesting - all walking through the door looking for wickedly good hospitality," Ellis said.
"The loyal customers have now become lifelong friends."
It was estimated, in the past 10 years, more than 250 people had been employed by the business.
"My crew are my champions, they are my business," he said.
Ellis said he and his brothers were going to take some time to decide what came next.