"Money had to be put aside each week to pay the essentials, like the gas bill. But we were always clean and tidy ..."
He was destined to see great things in his life, live through momentous times. For sure, his exploits on the rugby field all those years ago which culminated in his captaincy of the All Blacks, continue to define him for some. But Whineray's life has embraced an altogether broader canvas than sport.
Take his American experience. He went to Harvard Business School (on a Harkness Fellowship) from September 1967 to May 1969. It was an extraordinary time in the United States.
Martin Luther King and Bobby Kennedy were both assassinated, the Vietcong launched the Tet offensive against a background of growing, violent protests across America and police fired on students protesting against the war. Eventually, Lyndon Johnson announced he would not run for another term and Richard Nixon won the presidency.
Whineray has a curious anecdote from that era. "I remember one of the heads at Harvard said to us one day, 'We don't want any of you to feel that somehow you are beholden to us and you must love America for the rest of your days. That's not the case.
'We just want you to see if you can understand us - the way we are, the way we think, the way we get driven so that you can better understand the decisions we make and why we make them'.
"That experience definitely taught me a lot about the Americans. So, when I was in London at Lloyds Bank one September in 2001, having a meeting in the Treasury Dealing Room and someone rushed in and was just speechless through shock ... a plane had hit one of the Twin Towers in New York ... I remember my first thought was, someone is going to get it over this."
Yet he still doubts he understand Americans. But then, who does? After all, he smiles, "I'm not sure I understand New Zealanders".
He goes on: "When you look at American politics and see these extreme people, you think, 'Are they members of the same nation'? But I do have a better feel for them and their country now having worked there.
"When you mix with most Americans, they are nice people. But as for the extreme ones, you think, 'What planet are they on'?" He went there because he wanted to work with a global company. And he concedes the international experience changed him. "I had an awakening and awareness of trade and business in other parts of the world."
He smiles, the dappled sunlight that dances across his face illuminating the wrinkles of a 76-year-old man content now to ponder his life and times. "At Harvard in my second year, you had a choice of certain things to study. We were introduced to something called a computer and I thought, I have heard of those things, they might be around for a while. So I will take that course.
"There are so many advantages to this age of technology. Governments can't lie so much nowadays and better things may come. After all, the human race has been living better and better for thousands of years. Why would that stop now?
"At the turn of the last century, the average life expectation in the US was 47. Now it's nearly double that."
There is and always has been a pleasing ambience to Whineray's nature. Whether it was in his rugby or business days or now in retirement, he always demonstrated courtesy, had time for a word with everyone.
But as he surveys life from this juncture (not to mention the harbour from his comfortable apartment in the suburbs), is he happy?
"I am absolutely a contented man. Sometimes I wonder how it all went so well really."
But he equates that with specific memories, such as the death of his youngest brother Murray in an air crash in 1967.
And as he muses, "Perhaps it was all a bit simpler then compared to teenagers' lives today. Certain things are important, like education for young people. But also parents. I had a Dad with a good mind and a firm, strong mother."
Someone once called New Zealanders "a singular people" during the war years, which was an interesting description. But would it be true today?
"That sums up that generation, yes. And I would like to think it would still be true today. In fact, I don't doubt it. I think when the pressure comes on people do find another 'self' in them. They get fitter, stronger and harder. So I would like to think ... cometh the hour."
And finally, what of New Zealand? "We live by overseas trade both export and import so we have to continue to be competitive.
"But whatever we do, we are still victims of the great events overseas, like a cork in the waves, so it is important New Zealand retains strong links with the Western world, with its old friends."
It goes without saying, he personally has a great many of those.