School boards should be allowed to opt out of control by the Ministry of Education, and be bulk-funded according to the number of students they can attract, he said.
School boards that wished to stay under Ministry of Education direction could choose to do so, Mr Whyte said.
"However, I expect that a large portion would choose to be free. And that we would see dramatic improvements in the performance of schools, especially those teaching children from poor families."
The party also wanted to slash the number of bureaucrats working in the Ministry of Education, and give the saved money to schools.
The announcement has alarmed the New Zealand Education Institute Te Riu Roa (NZEI), which has called for Prime Minister John Key to rule out any coalition deal with ACT.
Voters in the upcoming general election would "rightly dismiss ACT's latest education policy as crackpot nonsense, and Mr Key should too," NZEI president Judith Nowotarski said.
"The charter school experiment has been dogged by controversy in its first year. No one is queuing up for more.
"Opinion polls show parents have no appetite for this experiment being rolled out any further."
If National wanted to maintain any credibility with parents and the teaching profession over its commitment to quality public education, it should immediately rule out acceptance of such a policy under any future coalition deal, Ms Nowotarski said.
"Charter schools have not lifted the educational success of the US, UK or Sweden -- on the contrary, New Zealand has consistently out-performed these countries."
Tackling inequality, not creating more competition, was the biggest challenge for New Zealand's system, she said.
"ACT's policy is a dangerous diversion from the real issues of how we invest in every local school to ensure it's a great school and how we support every child to get the education they need."