Mr Seymour said that it was by no means the most important issue facing the Government "but if it's worth doing, it's worth doing well".
If the Government wanted to replace one of the four finalists with Red Peak, it could do so by Order in Council - just a flick of a pen.
But if it wanted to add a fifth alternative, it would need to change the law setting up the flag referendum.
Labour's policy is to support a review of the current flag but it decided to oppose the two-stage referendum proposed by Mr Key as a waste of money.
Mr Key is wary of changing the process, but has not ruled it out altogether.
When asked about the popular alternative at his post cabinet press conference today Mr Key said he would only consider taking a bill to Parliament to add Red Peak to the ballot only if had cross-party support (excluding New Zealand First) and only if Labour actually supported the flag referendum process.
"They would need to go back and change their position on the flag process. Instead of lying to the public saying they are opposed to this when their policy is actually to change the flag, if they want to treat the whole process with respect they are welcome to come and have a discussion with me but that is not the way they have played this thing," Mr Key said.
"They have not played it with a straight bat."
Labour leader Andrew Little tonight threw cold water on any prospect of cross party support for adding the Red Peak flag to the ballot.
He said he would support a referendum process with integrity.
"Right now it is not one with integrity and there are tens of thousands of New Zealanders who are asking for a better process."
He said Labour would support a law change only if the Prime Minister agreed change the first referendum to ask voters if they wanted a change in the flag.
"I absolutely support adding Red Peak to the selection of flags if he is prepared to invert the order of the referendum questions," Mr Little said.
"That meets not just the concerns of those Red Peak supports but those tens of thousands of others who are expressing concern to us about the flawed nature of the referendum, being the order of the questions.
Mr Little said he personally quite liked the design of Red Peak but added: "I wouldn't be doing my job if I just said 'oh well, here's an opportunity now for Red Peak.'"
Cabinet accepted the recommendations of the Flag Consideration Committee as to which should be the four alternatives put to the vote in the postal referendum from November 20 to December 11.
The most popular of the four will be pitted against the current flag in a second referendum from March 3 to 24 next year.
A petition urging Red Peak to be added to the ballot is expected to presented at Parliament tomorrow.