The working group expected it to raise $8 billion over five years.
The Government will provide its response to the recommendation, among others made by the group, in April.
MORE:
• Comment: Proposed capital gains tax will leave Kiwi business owners out to dry
• Peter Beck slams Capital Gains Tax, but other entrepreneurs see positives
• Matthew Hooton: Michael Cullen's Capital Gapital Gains Tax is dead on arrival
• Capital Gains Tax will do more harm than good, bosses say
But the Real Estate Institute of New Zealand (REINZ) said a capital gains tax (CGT) would have a "punitive impact on the investment sector".
"I would expect more investors to leave the market because of this change because they just don't want to pay the CGT," REINZ chief executive Bindi Norwell said.
She said that this would, in turn, lead to a drop in house prices as it means there would be more houses on the market.
"That means there [would be] more supply which may push prices down slightly."
Norwell said this would occur in the short term as investors leave the market before the CGT comes into effect in April 2021 – if the Government decided to make it law.
In fact, the Tax Working Group itself said its proposed CGT would lead to "some small upward pressure on rents and downward pressure on house prices".
According to the group's recommendation, land and investment properties would be subject to a CGT but the family home would be exempt.
Neither Inland Revenue nor Statistics New Zealand have data on the number of New Zealanders who own a second property, so it is hard to say exactly how many people would be affected.
But data from CoreLogic revealed that 10 per cent of the 85,000 residential properties which were sold last year, some 8800 homes, were sold to people who owned another property.
That figure was 8700 in 2017, 12,000 in 2016 and 12,600 in 2015.
CoreLogic head of Research Nick Goodall said the extension of CGT for residential properties would further reduce the attractiveness of investing in property.
He said it would also likely cause a reduction in demand in the property market.
"However, as the tax will not be retrospective, we don't see it causing investors to hurriedly exit the market due simply to the proposed CGT changes."
National finance spokeswoman Amy Adams said the number of homes that would actually be stung with a CGT was not high enough to move the market.
NZ Property Investors' Federation executive officer Andrew King said it was difficult to say at this stage what impact a CGT could have on the property market.
But he said other in countries which do have such a tax, it hasn't had any effect on property prices.
"Anyone that thinks a capital gains tax would be the grand elixir to get property prices down is sadly mistaken."
Asked for comment, Finance Minister Grant Robertson said there was a "range of options" the Government was concidering.