At its nub the argument comes back to the margins banks are charging on retail rates. So ultimately it is about how much profit banks should be making.
Clyne says the debate is one he is prepared to engage in, if just to inject some balance into the arguments.
"It is fair to say we don't get a lot of third party advocacy," he jokes.
"I absolutely accept that $5.5 billion profit is a large number," he says, referring to latest National Australia Bank annual result.
But the issue of what constitutes reasonable profit is complex and is therefore a difficult topic for the modern media to process.
It's not the sort of thing that fits easily into a 140-character tweet, he says. He then proceeds to mount a defence of bank profits with the Twitter-sized statement: "The only true measure of profitability is return on equity. Australian banks fall about halfway through the top 100 companies."
That profit, he adds, is made off an asset base of about $500 billion.
Thorburn - a fellow Aussie who has headed the BNZ since October 2008, when he took over from Clyne - agrees.
"Margins have come up since the global financial crisis," he says.
"But margins are still generally on a long-term decline. If you want a strong banking system you have to have profitable banks.
"BNZ has 5000 staff over 200 outlets. To do that you need to have a margin that enables you to reinvest in that sort of thing."
In Australia the case for banks to maximise profits is easier to make because almost every member of the workforce has a stake in the banks through the compulsory superannuation fund.
"Given the huge penetration of super in the Australian market ... every working Australian is connected in some way to those profits."
Clyne says he often finds himself arguing with people who want to see banks make lower profits but still want them to pay strong dividends and offer the same high level of security.
"It just doesn't work like that," he says. "I'm happy to have the debate but what I always argue is that there are consequences.
"We can have lower profits but that may mean lower lending ... it will mean lower dividends."
If Australasian banks were to halve profits then they wouldn't be able to keep their AA investment ratings and that would add to the cost of borrowing for Australians and New Zealanders, Clyne says.
"Banks are managing some very delicate risks," says Thorburn.
"Liquidity risk, credit risk, maturity risks. Being profitable and building confidence of investors is vital to building a stable financial system."
The Australasian system stacks up exceptionally well, says Clyne, but the standout would be the Canadian system.
"It has this feature of five major banks. Which seems to be about the the right number to generate sufficient competition but also allow them to be of sufficient size to be stable and make investments."
When the Australian banks were downgraded from AA to AA- rating agencies highlighted the fact that Canadian banks don't rely on overseas funding.
So the Achilles heel of the Australasian banking system, and by proxy the New Zealand banking system, is the reliance on overseas money, Clyne says.
There is direct pressure on the Australian and New Zealand banks to move towards self-funding. The Basel III banking accord requires them to do so by 2018.
Australia's banks are about 65 per cent self-funding at present.
There are two ways to get to 100 per cent, Clyne says. One would be to cut lending - which would have drastic consequences for the economy. The more preferable option is to move slowly and steadily by growing the local bond market.
In reality New Zealand will always struggle to get to 100 per cent local funding, Thorburn says.
"In New Zealand we need to save more," he says.
"I don't have a problem with New Zealand financing some of our growth through offshore markets ... we're a trading nation.
"But we do need to get our 65 per cent to 80 per cent.
"This is where there are tremendous advantages for the linkage between he New Zealand and Australian banking systems," says Clyne.
"Under Basel we have to get to 100 per cent as a group. So if New Zealand only has the capacity to get to 80 it's far better it's in the group."
