The house is the former flat of Kiwi band Six60, who formed and regularly played gigs there.
Flames could be seen spewing from the front door of the flat this morning as a thick cloud of smoke engulfed the area.
One of the flatmates, Oliver Thode, was lucky to have been staying across the road when the fire started.
"I'm just a bit shocked - how do you feel after a fire?"
Mr Thode had visited his three injured flatmates in hospital. He said his seriously injured friend was about to undergo surgery.
"He jumped out of a window to get out and so he broke his ankle."
He believed the other two men were "fine".
Mr Thode said he was staying across the road from his house when the fire started and was woken by a flatmate who had raced over to check he wasn't in the blaze.
"Everyone was asleep when it started I think."
He suspected the fire may have been started by a switchboard beside a stairway in the house.
Mr Thode had not seen the full extent of the damage to the house yet, but believed many of his possessions had been ruined by smoke and water.
"Hopefully I'll find somewhere new to live soon."
A woman who lives down the road from the house said she was woken at 6.45am by panicked shouting in the street.
"My flatmate heard them shouting 'help, help'."
The woman said her flatmate called 111 and the whole flat went out into the street to see what was happening.
"There was a big flame coming out the front door, there was smoke everywhere.
"We ran out on to the street and the boys from 660 were standing outside the house smashing the glass and shouting someone's name like they thought someone was in there or something."
Castle St resident Clara Lewis said watching the fire was sobering.
"Our whole flat went out, it was so sobering because at that stage everyone was crowded around watching the flames. I'm so shocked."
Natalie Lewis said she was woken by the sound of smashing windows and people screaming.
"It was really quiet last night and then this happened. It was pretty scary to wake up to."
Two fire trucks were sent to the scene from Willowbank after the Fire Service received multiple calls. The fire was quickly brought under control.
A firefighter injured his knee at the scene and had gone to hospital to have it checked. However, the injury was minor.
East Otago area fire commander Laurence Voight said the blaze could have been fatal.
He said new smoke alarms had been installed by the landlord in January. However, two of them had been removed and were found in cupboards.
"Smoke alarms can't do their job if they're not on the wall to do it. Taking them down is an act of stupidity," Mr Voight said.
"This is a lesson to anyone else who's made a short-sighted decision to take theirs down. Working smoke alarms would have given them more warning and more time to escape."
Fire investigator Cathy Trevathan was at the scene but it was too early to say what had caused the fire.
Six60 tweeted that the fire looked bad.
"This looks bad. Hope everyone is doing OK. Send us any info."
Both Six60 and Castle St hit headlines in March when a balcony collapsed during the band's annual gig, injuring 18 people.
Up until this year the band held an annual gig at their old flat, but due to limited space they moved their show this year to 598 Castle St.
