Credit card repayment insurance covers some or all of a person's credit card repayments in certain circumstances such as redundancy, bankruptcy, injury, illness or death.
Today, the court heard the issues had emerged through the merger of ANZ and National Bank nearly a decade ago.
But the FMA's counsel Nick Flanagan said such breaches were occurring years before.
"They go back to the 90s," he said.
The FMA's claim, however, only reflected the period since the introduction of the FMCA which came into force in April 2014.
It is also the first civil proceedings the FMA has brought under the fair-dealing provisions in part two of the legislation.
Justice Muir said the issue he needed to grapple with was whether ANZ should've had such robust systems that such breaches didn't continue for years.
"Everyone accepts that these were unintentional breaches," the judge added.
"If it's systems were as robust 10 years ago as they were three years ago then they would have been picked up earlier, is that fair?" he asked.
Flanagan said ANZ and other banks must operate systems "so that these things when they fall between the cracks don't last for 20 years".
Justice Muir also said there was no deliberate intention from ANZ to withhold information from the market watchdog.
Flanagan, however, told the court ANZ had decided not to tell its customers.
But Justice Muir felt doing so may have created a "level of anxiety and a significant management issue" which could have become larger than the problem itself.
Both the FMA and Justice Muir were critical of how long ANZ took to flush out the problem and repay its clients, the majority of whom were dealt with fairly and appropriately.
A sum of $680 on average was not insignificant, Justice Muir added.
ANZ's counsel, Andrew Horne of MinterEllisonRuddWatts, told the court the bank didn't want any of its customers paying for a non-existent product.
"When you are a large bank and you have many customers with many different interests," he said.
He added it was a difficult and complex process to identify the issues and pay back customers, some of whom also needed to be refunded paid interest.
But Horne admitted the bank took too long to deal with the problem once it became aware of the issue.
"It took time to chase rabbits down holes and identify what had gone wrong," he said.
"This was not a straight forward issue."
ANZ earlier said it self-reported the issues to the FMA in June 2019 and had already compensated customers $440,000.
Horne also said the issue may have been identified earlier if one of the affected customers had noticed the irregularities in their bank statements.
However, Justice Muir was unsympathetic to such an argument and said the same could be said about the bankers.
The judge reserved his decision but said he was prepared to endorse the joint recommendation in regards to a penalty of about $400,000.
He said deterrence "always sits in the background" but may not be a factor in ANZ's case.
The FMA had sought both a pecuniary penalty of $280,000 and declarations of contravention.
It is expected the judgment will be delivered in a few weeks.
