"Until a few days ago, we had not been furnished with any evidentiary material from any of the parties in this dispute. However substantive material has now come into my possession, some as recent as 2.15pm this afternoon.
"The information we have received leaves me in a position where I have no confidence in Mr Horan's ability to continue as a member of Parliament, and he will be expelled from the NZ First caucus forthwith."
Mr Peter's decision came as a surprise to Mr Horan, with his lawyer Paul Mabey, QC, saying he was unaware his position in the party was at risk.
In an interview with Radio Live late this afternoon, Mr Mabey said Mr Horan has been in regular contact with Mr Peters up until last night.
"But he had no idea this announcement was being made and he certainly didn't know that his position in the party was at risk and he doesn't know what the information Mr Peter has.
"He says he's got information to justify his decision, which may be true but Brendon Horan doesn't know what's behind these allegations. It's from a family member...but whether that information is true is another matter."
"Mr Horan's information is only as good as yours or mine. He goes on what's in the media - no one's put an allegation to him directly."
In a later statement, Mr Mabey said Mr Horan had "no intention" of leaving Parliament.
"He has no reason to do so. Mr Horan is the subject of unproven allegations which he completely denies. None of the allegations have ever been put to him directly nor has he shown any evidence to support them
Mr Peters told reporters after his statement to the House he was bitterly disappointed at how things had turned out.
"You bust your gut. You make some serious sacrifices and so do hundreds and thousands of others and it's always a bitterly disappointing moment."
"But we have got a responsibility to the public of this country, to Parliament and to the party itself.
"My party will be relieved that we have done our duty."
Mr Peters said that the caucus had been briefed on Mr Peters' findings but it would meet later today to expel Mr Horan. His expulsion from the party would be automatic.
He would not discuss the details of the information he had or why he had come to his decision.
Asked if he believed it was enough to warrant a criminal investigation, he said he would not comment.
He said he had used parliamentary privilege to make his statement because "I am not going to be subject to people spraying defamation writs that cost you a fortune, no matter how correct you might be or not and that is used to muzzle people. I did not wish to expose myself to that."
He said issues of whether Mr Horan had a mandate to continue in Parliament, seeing as he had come in on the list, should be addressed to Mr Horan.
But he alluded to a bill passed by Parliament in the wake of party hopping in the period from 1994 to 1999. It forced MPs who had been expelled from their parties to leave Parliament but it had a sunset clause and is no longer in effect.
"We supported the waka-hopping bill and we didn't ever support that it had a sunset clause," he said.
"We supported it then and we still support it."
Mr Peters agreed he had been put in the position of judge over Mr Horan ''because that's position I'm in and I have got to take it deadly seriously as you all did in the last few weeks".
"I did and when I got the information I treated it seriously."
Prime Minister John Key, when asked if Mr Horan had lost his mandate to continue as an MP, said Mr Peters had made his announcement under parliamentary privilege and it would be foolish of him to comment.
Earlier today, Mr Horan's lawyers issued a statement on his behalf, saying he rejected any suggestion he had misappropriated money and was confident a full investigation would exonerate him.
The suspension does not mean Mr Horan has to leave Parliament. He can stay on as an independent MP. Mr Peters said he believed Mr Horan was honour bound to leave Parliament altogether.
- additional reporting nzherald.co.nz