Mr McCallum stood for Act in Dunedin North in 2011 and has been a member of Act on Campus since 2010 and its vice President since 2012.
He told Critic he was "blindsided" by Dr Whyte's comments but had been given some indication the party leader was planning to court controversy.
"Jamie was in Dunedin for a small gathering of ACT supporters on the morning of 20 July, and he mentioned to me that he was in search of a 'stunt ... because you know, the polls.' A week later ACT rolled out a controversial and obviously unprepared race relations policy."
Mr McCallum told
Critic
Dr Whyte's call for Race Relations Commissioner Dame Susan Devoy's resignation after she criticised his speech as "grotesque and inflammatory" was "the final straw".
"People criticising Jamie for One Country, One Law have only been met with derision; either they haven't read his speeches, have ulterior political motives, or are, by him, wrong."
He said the speech and other bids by Dr Whyte to raise Act's flagging poll ratings were inconsistent with his own beliefs.
"Act's policies are about reminding you of scary burglars, zealous bureaucrats with a hidden green agenda, and resentful Maori of taking an equal placing. This is the imagery the vague words are designed to create. Liberals and libertarians are getting a rough deal from ACT.
"The positive ideals they represent are used as currency to hedge conservative influence, like the Tea Party. A lot of libertarians have avoided ACT for these reasons. For wanting to change that, I find myself in present circumstances."
Mr McCallum said he had spoken to other members of the party "who are becoming concerned that ACT is focusing more on fear and prejudice to gain votes".
Dr Whyte told the Herald Mr McCallum had indicated to him that he wasn't happy with the speech so his resignation was not a surprise.
However Dr Whyte said he was surprised that Mr McCallum had gone public with his criticism of the speech and policy.
"I like Guy. He's a very nice guy. I was honestly confused by his response because this is a long standing Act policy, but there you have it and he's resigned.
"I think that's probably a wise thing to do if he doesn't agree with the policy because I think it's quite an important one."