The 79-year-old Karajan "had difficulty walking and I would help him" Jo recalls. "We talked a lot - me about my boyfriend, him about his daughters, his cars and his passion for technology. I miss him a lot."
But she has taken the maestro's advice to heart. "It's not an easy life," she says.
"You spend hours studying, memorising notes and lyrics, often in different languages. Then you have to travel all the time, far away from home. And all the time you just think about your throat.
"There's no pleasure," she laughs. "It's the life of a nun."
Yet there is a real thrill in giving audiences pleasure, having them "open their mouths and go 'wow' because they can't believe the human voice can be so like an instrument".
Which is what happened when she last sang Adolphe Adam's virtuoso variations on Mozart's Ah! Vous dirai-je Maman, which she delivers for us next week, alongside popular arias by Gounod, Verdi and Offenbach.
"There's a little bit of everything," she says. "French, Italian, Leonard Bernstein's Glitter and Be Gay, some Korean songs and then some German.
"I'm a rainbow," she exclaims. "Sometimes I feel like an 8-year-old girl seeing the world in all its innocence; at other times I'm an 80-year-old woman who knows the worries of the world too well. My moods can change 20 times a day - but then I'm an artist." Before she discloses her favourite arias, she confesses that her latest bete noir is the storm of coloratura that Mozart wrote for the Queen of the Night.
This has been her calling card for years and she laughs at how it turned up in the movie Eat Pray Love. "Julia Roberts is in Italy eating spaghetti and in the background there am I singing Queen of the Night."
She giggles slightly too at being "the voice of the Devil", vocalising Wojciech Kilar's theme for Roman Polanski's The Ninth Gate, "a really scary movie".
More recently she is on the smaller screen in Todd Haynes' HBO mini series Mildred Pierce which debuts on Sky's Soho from next Friday. Haynes uses a number of Jo's arias as well as a new recording of I'm always chasing rainbows ("an amazing song which I loved recording in New York").
"The story of the drama is quite similar to mine," Jo points out. "My mother wanted to be an opera singer and, when she couldn't, she devoted herself to my becoming one. She was very strong and ambitious.
"She used to sit in the front row and watch me very carefully. I'd tremble in my dressing room waiting for her judgment. Sometimes she'd come in and just talk about the weather or something which would mean she didn't like what I'd done. But when she was happy, she really energised me and gave me the courage to go on."
Jo has said that these days she prefers wily comic heroines like Marie in Donizetti's La Fille du Regiment to the composer's tragic Lucia. "She does not have the strength to resist the pressures of society and just dies. It is interesting musically, but the weakness of these characters makes me feel a little bit sad."
Jo admits, too, to being more than a little sad herself, separated from her three beloved dogs back in Italy. This brings up another tinge of regret - that, having devoted herself to a musical training, she was not able to study to be a veterinarian. "Of course I'm happy to share my talents and my voice," she protests. "But, in my next life, I want to be an animal doctor."
Performance
Who: Sumi Jo
Where: Auckland Town Hall
When: Wednesday at 7pm