"I need the facts in a very serious allegation, and we do take it seriously.''
He would not say if he had confidence in Mr Horan.
Mr Peters said he had asked for evidence two months ago, after Mr Horan's brother had phoned him but he had received no evidence to date.
"I did leave that conversation on the basis that he would get me the information and pronto,'' said Mr Peters.
"This is a priority matter and it's urgent.''
Mr Horan was on leave until Mr Peters had viewed bank statements and other evidence.
He confirmed he had been given strict instructions by Mr Peters, but he would not say what those instructions were.
"That's between Winston and I, and I have no comment to make,'' he said.
When asked when he would be back to continue his duties in Parliament, Mr Horan replied: "if the media would leave my family alone it would be as expeditiously as possible.''
He would not say why there had been more than two months' delay in providing information to Mr Peters.
"I'm not the person holding things up here,'' he said.
"It's a personal family matter. It should never be in the media spotlight.''
Mr Ormsby said the bank statements showed a large number of withdrawals from TABs and from ATM machines near Mr Horan's office in Mount Maunganui, Auckland and Hamilton.
They also showed numerous payments at takeaway food outlets and video shops.
Mr Ormsby has said his initial investigation suggests about $85,000 was missing from his mother's account.
Mr Horan has said firmly he will not step down from NZ First because of the matter. He called in lawyers after an article was published revealing the allegations.
Mr Horan's mother, Olwen Horan, 87, of Mount Maunganui, died in early August about three months after signing a codicil to her will, after becoming concerned that money was missing from her accounts.
Earlier today, Mr Horan said he would not step down from New Zealand First in the wake of the allegations.
The former TVNZ weatherman and surf lifesaver has dismissed claims he had anything to do with the missing money as "fantasy".
The New Zealand First MP has also brought in lawyers after an article was published revealing the allegations.
When asked if he considered stepping down from his party while the matter was being dealt with legally, Mr Horan replied "no, of course not".
"I can't comment anymore because it's in the hands of the lawyers now," Mr Horan said.
"I'm sure you can imagine what it's like."
Mr Horan said he did not know how long it would take before the matter was resolved.
"These things, I guess they take time."
