This newspaper's analysis shows 41 properties at Bridesdale are owned by building professionals - building companies, developers or house rental firms.
Labour's housing spokesman Phil Twyford says special housing areas have been a "spectacular flop".
"Most of the benefits of the special housing areas have gone to investors and speculators."
But Building and Construction Minister Nick Smith calls for patience.
"What we know is that if we keep the foot on the accelerator, of growing supply, you'll get supply and demand back in balance and that is when you'll get housing prices stabilising."
However he warns that, for Queenstown, that might take "three or four years".
Queenstown councillor John MacDonald, who chairs the mayoral housing taskforce established in April, says it would be disastrous for Queenstown if property prices continue to rise and nothing is done.
He wants to cut out the government.
"We're going to have to come up with some local understandings and some local solutions."
Queenstown Lakes Community Housing Trust boss Julie Scott is disappointed but not surprised first-home buyers don't own the majority of Bridesdale plots.
The fast-tracked subdivision was approved without any mandate for affordable housing.
Subsequent special housing areas will make contributions to the trust but details are confidential, Scott says.
The trust's waiting list has almost doubled to 450 since mid-2014.
"We can't keep up with the demand," she says. "It's great that we are getting contributions coming in from SHAs, but these things take time. None of them are built and complete yet."
Minister Smith blames Queenstown's situation on high section costs - with some plots of about 500 square metres selling for well over $300,000.
"That is too expensive," Smith says, adding the solution is "persuading the council to free up the additional land".
The government wants to take over Wakatipu High School's site, which will be vacated at the end of the year.
Twyford says a Labour-led government would consider owning a stake in a worker accommodation development or owning the property - working with the private sector and council.
"The market on its own is simply not going to fix this problem. That's why key workers in Queenstown can't afford to live in the community."
Business Ministry data now put mean Queenstown rents at $581 a week - higher than $557 charged by landlords in Auckland city and North Shore.
Queenstown Lakes is also the least affordable place to buy a house.
The median house price of $880,000 in May is now 12.21 times the median household income of $72,098, according to Interest.co.nz.
Auckland's North Shore is at 10.55. The national average is 6.21 times.
Builder owns nine
Queenstown builder Andrew Campbell owns nine Bridesdale sections - one personally and eight through two companies.
He says they're not for him, they've been pre-sold to clients for home-and-land packages.
Packages in the fast-tracked subdivision were originally marketed at $450,000.
But Campbell admits: "Every year the price goes up."
Two-thirds of his clients are out-of-towners.
"They've indicated to me that they want to get a foot in the door in the Queenstown market and use it for either a holiday home, which is why they want to keep it to a more affordable level, or they may look at it as a rental.
"All we're doing is providing a service for a demand that people want."
