"But they didn't have a strong enough framework about what to do about it. It sort of fell apart, which was a shame," he said.
In New Zealand it is tempting for people with a vested interest in the status quo to question the relevance of these concerns here. And there is little doubt that inequality is less extreme here than in the United States. But that does not set the bar very high.
Yellen said that in 2013 the lower half of American households, ranked by wealth, held just 1 per cent of all the wealth, while the richest 5 per cent held 63 per cent.
The nearest comparable New Zealand data - unfortunately 10 years old - shows the lower half of the distribution owning 5 per cent of households' total net worth, while the richest 5 per cent of households had 38 per cent. It is unlikely that things have moved in a more egalitarian direction since then.
The big increase in income inequality occurred between the mid-1980s and the mid-1990s. Since then high-level measures of income inequality like the Gini coefficient have wobbled about a flat trend and we have ended up in the middle of the pack of developed countries.
But that also means that we have had 20 years for this level of income inequality to accumulate and harden into disparities in wealth.
A particular concern Yellen highlighted is the relationship between wealth and educational opportunities, ranging from access to early childhood to tertiary education.
In that respect the recent report from the OECD should give us pause, too. It found it is getting easier to predict how well a 15-year-old Kiwi kid will do in the international PISA maths tests by looking at his or her socio-economic background. There is a warning light flashing there.
And when you look at the impact of housing costs on household finances it is not hard to see why.
Just over one in four households in the bottom fifth of the range, when ranked by income, reported spending more than half their income on accommodation last year, up from one in five between 2004 and 2009 and higher than at any time since this statistical series began in 1988.
This is the underbelly of the cheerful news we had from Credit Suisse last week that New Zealanders' net worth has quadrupled since the year 2000.
Two things explain that eyebrow-raising result: there has been a lot of house price inflation since 2000, for one thing, and Credit Suisse uses the exchange rate of the day when doing its sums, so the figures are inflated by the fact that 14 years ago the New Zealand dollar was worth US40c, about half its current level.
According to Reserve Bank data, households' collective net housing equity (the lion's share of household wealth) in 2000 was $166 billion. By the end of last year it had risen to $530 billion. Some of the increase is real but much of it is just inflation.
The 2000 figure converted to US$66 billion at the time; the 2013 one would be US$436 billion.
However, the opportunities to realise or withdraw some housing equity are limited and the rest of the time it is just paper wealth, while the plight of those not on the housing ladder is well known.
New Zealand is not a "rock star economy" but if there is such a thing as a rock star economist it is the French scholar Thomas Piketty.
He makes a powerful case that - absent wealth-destroying calamities like world wars or the Great Depression - in capitalist economies wealth will become more and more concentrated in fewer and fewer hands, unless policymakers intervene.
A valuable contribution to the domestic debate is a book, The Piketty Phenomenon, published yesterday, whose 15 contributors, mainly economists, reflect on whether his findings hold good for New Zealand, and if so whether it is a problem and, if it is, what can be done about it.
The short answers are, respectively: pretty much, yes and that's a tricky one.
But stepping back from that it is clear there is more to the inequality story than Piketty's thesis about the internal logic of capitalism.
The increase in inequality in rich countries since the 1980s has coincided with a reduction in inequality globally as huge numbers of people in the emerging market economies make the move from abject rural poverty to something closer to what we would consider a decent standard of living. It is not a zero sum game but there has been an element of making room for them.
And there is the impact of disruptive technological change, destroying traditional business models and livelihoods at the same time as it opens up all sorts of new opportunities. Healthy and Schumpeterian it may be, but not without casualties.
The risk is that we get into a sterile debate about which to focus on: "That's not the problem. This is the problem." We are allowed to have more than one problem at a time.