"The Education Ministry, Talent2 and the Inland Revenue Department are working together on this," said Mr Joyce.
APNZ has asked Mr Joyce and the Education Ministry for details of all overpayments, underpayments and non-payments of income tax, student loan repayments and KiwiSaver payments. They say the information does not yet exist.
In February the Government agreed to put aside up to $5 million to help deliver stable pay periods, clear the backlog of outstanding pay issues through the backlog Clearance Unit and resolve system bugs and defects.
In February there were 613 bugs, of which 60 were considered as very serious.
The technical review said there should be fewer than 10 very serious bugs with the Novopay system by the end of June - there are now eight.
There are still a total of 316 bugs.
Since the establishing of the Backlog Clearance Unit, 12,000 pay changes have been administered.
Mr Joyce said work was still needed on work balances, clearing overpayments and resolving outstanding tax issues.
The latest independent assessment of school pay rounds administered through Novopay show mistakes in the pay have been maintained at a level required by a technical review earlier this year.
Earlier this year it was recommended the error rate with each pay cycle should be 0.5 per cent or better.
Eight of the last 10 pay rounds have been maintained within that range.
PricewaterhouseCoopers have been assessing the number of mistakes with pay - they have assessed 12 pay rounds.
Last year the error rates were as high at 2.2 per cent, but in the latest pay period on July 10, 0.22 of school staff were incorrectly paid - 13 staff were notified as not paid, 91 were overpaid, and 97 underpaid.
"This is pleasing to see given Pay Period 8 was the largest payroll so far this financial year and had the smallest number of issues of the 12 pay periods that have been monitored since February," Mr Joyce said.
Complaints received from staff about their pay:
2013/14 financial year:
Pay period 8 - 0.22 per cent.
Pay period 7 - 0.26 per cent
Pay period 6 - 0.30 per cent
Pay period 5 - 0.39 per cent
Pay period 4 - 0.42 per cent
Pay period 3 - 0.26 per cent
Pay period 2 - 0.44 per cent
Pay period 1 (April) - 2.14 per cent
2012-2013 financial year:
Pay period 26 - 0.43 per cent
Pay period 25 - 1 per cent
Pay period 24 - 1.9 per cent
Pay period 23 - 2.2 per cent