He indicated schools should brace themselves for the worst. The next pay period was expected to be the system's toughest test with changes to pay rates as a result of the new secondary school teachers collective agreement.
"The last three pays have been quieter because of the school holiday period, the big challenge is getting through the next pay cycle on February 6, and playing catch-up," said Mr Joyce.
"It's like trying to change the tyre on the car when you're driving along the motorway at 100km/h."
Mr Joyce was speaking after meeting principals and staff from Wellington Girls' College and Northland Primary School in Wellington today.
College principal Julia Davidson said Mr Joyce met two staff involved with administering the system.
"I approached him because I thought possibly he was seeing it from an individual case, but I think it's important he sees what is happening in a school and the [amount] of time that my staff are having to take to do this."
Northland principal Jeremy Edwards said many of his staff had been overpaid by up to several thousand dollars.
Staff were waiting for letters claiming to claim the money they were overpaid back.
"The process is taking a very long, long time," said Mr Edwards.
Post Primary Teachers Association president Robin Duff said he hoped problems wouldn't occur as a result of the new collective agreement.
"When Talent2 tendered for the contract they surely would have known there would be pay increases," said Mr Duff.
Labour acting education spokesman Chris Hipkins said it was a cop-out to say schools had to prepare themselves for more problems.
"Steven Joyce needs to put a contingency plan in place urgently so that if Novopay still isn't delivering in two months' time he can kick it to touch," said Mr Hipkins.
New Zealand First education spokeswoman Tracy Martin said the warning would be cold comfort for thousands of school employees who have been struggling with Novopay for months and it was time ministers were accountable.
"Education Minister Hekia Parata signed off on the system, which came with no back-up plan, on the advice of her associate Craig Foss who has somehow managed to keep his place in Cabinet.
"It's been a litany of monumental errors for the Government, yet Mr Joyce insists on persevering with what is clearly a nightmare for the education sector."