Asked if he believed he had been treated fairly, he said he did not.
"How would you feel if you'd done nothing wrong and circumstances led to this?"
Mr Horan said he wanted to stay on and remained strongly committed to NZ First and its principles and policies. He would discuss with his family whether to stay on as an independent MP - should he be exonerated - if Mr Peters refused to take him back.
He was certain that he would be cleared on the allegations.
"But I have no intention of standing down. Why should I?"
Mr Horan said he was leaving most of the public comments to his lawyer, Paul Mabey, QC.
"Right now I'm more concerned with spending time looking after my wife and children.
"As you can imagine, it's a very stressful time, but I've done nothing wrong," he said.
Mr Mabey said yesterday that Mr Horan was blindsided by Mr Peters' decision and had not been given a right of reply or shown the information Mr Peters was relying on.
"We've seen no evidence of any wrongdoing by Mr Horan at all. If Mr Peters has, it would be good to see what he's got."
Mr Peters would not reveal the details of the information he saw and would not comment when asked if he believed it was enough to warrant a criminal investigation.
He said he had conducted an investigation after receiving information over the past few days.
It is understood that included financial and other records, as well as information relating to allegations that Mr Horan had a gambling habit.
He had requested evidence from Mr Ormsby, and Mr Horan to either prove or disprove the allegations.
Forensic accountants have been investigating Olwen Horan's estate since the allegations.
Yesterday police said no complaint had been received.
The will's executor, John Buckthought, refused to comment and Mark Hornabrook, Mark Hornabrook, the lawyer for Olwen Horan's estate, also refused to comment.
Mr Ormsby has previously said Mrs Horan's bank statements showed several withdrawals were made at TABs and ATMs near Mr Horan's office in Mt Maunganui, in Auckland and in Hamilton.
Mrs Horan died in August, soon after she signed a codicil to her will authorising her nephew, Mr Buckthought, to act as executor and "recover monies by any lawful means from my daughter Marilyn Bleackley and son Brendan Horan, which has been either loaned to them by me, or taken from me by misadventure".
Read more: NZ First blocks out 'Mr Sunshine'
- additional reporting by APN