The tax would be applied after sale on the gain in value between April 2021 and the sale price.
National Party leader Simon Bridges said the 403,883 figure included about 50,000 farms.
But it did not include those who ran a business from home, or who had flatmates.
"They would also be subjected to a CGT.
"The multi-million dollar home in Remuera and Oriental Bay will be protected from a CGT, but the three-bedroom house on two acres in Wyndham in Southland, listed on Trade Me with an asking price of $260k, would be included.
"Labour claims this is about fairness, but how's that fair?"
In a statement, Finance Minister Grant Robertson said: "The independent Tax Working Group's proposal was that CGT would be paid only on the sale of a property, and that all gains made up to the point the tax comes in would be protected – the proposal was that only gains made after 1 April 2021 would be subject to a CGT.
"Final decisions have not been made on responses to recommendations the report, and we will look at all advice on them."
The Government is expected to announce in April which of the Tax Working Group's recommendations it will take into the 2020 election.
Tax Working Group member and PwC tax partner Geof Nightingale has said the group chose 4500sq m "because that is the current definition of land that goes with a house that presents for GST purposes when for example, a farm is sold".
"The working group also said 4500sq metres was just a suggestion [when] you define what amount of land is reasonable for the occupation and enjoyment of the actual house. In some cases it might be less or might be more," Nightingale said.
"That is going to be an area of real interest in submissions if the Government takes forward any of this proposal. What is the family home and what land comes with it is going to be the debate."
Nightingale said using the Real Estate Institute of NZ's median lifestyle block size of 20,000sq m as an example, it would mean the house included on 4500sq m of land would not be subject to a capital gains tax.
"But the 15,500sq m round it would potentially be subject to capital gains tax on any gain that related to that land."