As a toddler introduced to a father fresh home from the sea, Paxman recalls simply screaming at the unknown face.
'Relations between us never really improved much,' he continues, describing a man who beat him at the slightest provocation with sticks, shoes, cricket stumps or his bare hands. Much later, writes Paxman, he came to forgive his father for his appalling temper - but even then he appears to sneer at what he describes as Keith's naivety in business and doomed aspirations to be a country squire attired in plus-fours and a monocle.
All of which has come as a rude awakening to 75-year-old Celia, who lived with Keith for the last 31 years of his life, and today still works as a physiotherapist in her native New Zealand.
She is both astonished at Paxman's antagonistic portrayal of his own father, and aghast at such a public demolition of the man she loved. Today she is determined to defend his reputation. She also questions the memory of her famous stepson, who still presents the University Challenge quiz show with his trademark languid disdain.
'Keith was extremely proud of Jeremy and didn't realise there was any great wound to heal,' she says. 'It's only now, reading these interviews and extracts from the book, that I realise what was lurking.
'It sounds quite vicious and ugly and it surprises me that Jeremy is now coming over so bitter.
'Keith enjoyed a bit of banter and could be a bit brusque. He called a spade a spade, but he was a warm and tender man. I would not have allowed a bully in my bed.'
It is true, of course, that Celia - Keith's second wife - did not witness his unhappy home life with his young family. And she acknowledges that her husband, a bluff Yorkshireman born in 1922, was a man of his time, more at ease with his Navy shipmates, and in his later career as a finance director and industrial troubleshooter, than baring his soul. He was no metrosexual softie.
But she insists that his feelings for all his children were genuine and profound.
'Jeremy may have been spanked - we were all spanked in those days,' she says. 'I was spanked myself with a wooden spoon and of course I remember it, especially the times that I felt were unfair.
'But Keith was a gentleman rather than a thug. He was very proud of all of his kids and their achievements. But it was difficult, maybe, for him to express it to them.
'He may never have hugged Jeremy or told him that he loved him, but I know that he did. Maybe he thought it was so obvious that it didn't need saying, and it's true that intimate relationships were something he was less inclined to work on than financial problems.'
So what could have prompted such a cold description from his own son?
Jeremy is the eldest of four children - the other three are Giles, a former British Ambassador to Spain; James, a conservationist; and Jenny, a retired BBC producer and barrister. They were already grown up when Keith effectively left them, their mother and Britain behind for a new life halfway around the globe.
Keith had already sailed around the world when, in 1974, he took up an invitation to run a training ship for young sailors in Auckland, New Zealand. And the Southern Hemisphere was where he stayed.
By 1979 he was working as finance director for an orange juice factory in the Cook Islands in the South Pacific, and that's when he met Celia, herself an accomplished yachtswoman,
'I had a 26ft sloop and had sailed to Rarotonga from Tahiti to meet my uncle,' she remembers. 'There was Keith on the wharf and I threw him the rope which he tied up with a fancy knot and I thought, "Hmm, that was smart".
'He asked me, "Do you have a dress on that tub of yours?" and when I said yes, he said he'd take me and my uncle out to dinner.'
Keith was 57, nearly 20 years older than Celia, yet it seems there was chemistry from the outset, as she is not afraid to admit.
'After dinner we dropped off my uncle, who had been an Anglican vicar, and I went back to Keith's house. We leapt into bed.
'It wasn't that he was dashing or athletic - he wasn't - and yes, he did wear a monocle back then. But that was for convenience, not snobbery. He was a just a nice bloke, kind and funny, and he fancied me like hell.'
Keith's marriage had broken down well before: 'I think he and Joan were sick of each other - she didn't want him back.'
After an idyllic ten days together on Tahiti, Celia sailed onwards to Fiji, but took up an invitation to return as soon as she was able. She and Keith were together from then on, and eventually set up home in rural Queensland, Australia.
It was there that she met Jeremy for the first time - her recollection of which forms the first of her many objections to Jeremy's memoir.
In his book, Paxman describes knocking on his father's door in the 1980s, seemingly out of the blue, and being met by a careworn man in an orange sarong who gave him a fumbled hug.
Celia's memory, however, is rather different. In fact, she recalls a much-anticipated visit from Jeremy, Liz, and their eldest daughter Jessica, then aged five or six, in 1997, and an occasion of warmth and enthusiasm.
'Keith had talked from the start about his life back home and his family, just to fill me in,' she says. 'He was in touch with all the children as far as he was able and he talked about them all the time. He thought they were wonderful, as dads do.
'He was excited to see Jeremy for the first time in a long time, even more so since he was bringing his wife and daughter.
'We had about eight acres at a place called Beenleigh between Brisbane and the Gold Coast, and they came and stayed with us for about two weeks, I believe.
'I wasn't aware of any great tension. There was lots of playing in the garden, picking fruit and family expeditions to the beach, and we also went on a ferry to some of the islands.
'I remember it as a happy holiday and it wasn't long after they got back to the UK that Liz found she was pregnant with their twins, Victoria and Jack.'
Paxman also says he met his father only once more, when the BBC sent him to Australia for work.
Here again Celia disagrees with the TV star. She believes that her husband met Jeremy and his other children on other occasions, when he made trips back to Britain.
It is true he never stayed at his son's picturesque home in Stonor, Oxfordshire; Celia, though, was a guest there when on a holiday in Britain without Keith. 'Their home is gorgeous and they seemed to accept me into their family,' she says. 'He took me to Oxford to show me around but I didn't get any feeling that he had hated Keith or couldn't bear him.
'He didn't make any attempt, that I remember, to understand him better or know more.'
Celia emerges rather better from the book than her late husband: Jeremy describes her as 'kindly'.
Indeed, she even helped with his research. In June, she emailed Jeremy with dates and details of his father's movements around the Pacific in the years since he had separated from Joan.
Jeremy received them eagerly, and sent them on to his younger siblings in case they and their children were interested.
And it is the subject of the grandchildren that seems to have hurt Celia most in recent days, and in particular Paxman's suggestion - in an interview to promote the book -that Keith had been uninterested in them. He described it as 'part of a life he'd put behind him'.
Celia's response has the terseness you might expect from Paxo himself. 'That's just bull****,' she says. 'He was always involved with Christmas shopping for the family and Jeremy's children would all have had presents from their grandpa. He loved getting progress reports about them and he and Jeremy would talk on the phone and have a bit of a giggle.
'But it was his brother Giles and his wife, and his sister Jenny who led the way in keeping in touch and sending us pictures and videos of their own kids and Jeremy's.'
It was Celia who rang Jeremy in March 2010 to break the news to him that his father had died after a three-week stay in hospital - he had been too ill to fulfil his dream of returning to Britain to be surrounded by his family.
All four siblings were involved in arranging a ceremony, chartering a boat to scatter their father's ashes in the Solent near his former Navy base in Portsmouth.
'There's a particular buoy where all the Navy officers get chucked over, so Keith went in there with all his mates,' says Celia, sitting in her husband's favourite armchair where, in a nightly ritual, they would toast each other lovingly with a glass of ginger wine.
'There were no formalities but we had a lovely day and the younger kids all had a great time throwing his ashes into the wind and seeing them blow back over everyone.'
As a parting salvo in his book, Paxman claims that the four children's joint inheritance from Keith was a mere £140.22. In fact, says Celia, although he died with few possessions, he had already given all his grandchildren a far more meaningful token of love.
Keith worked for British Steel for a time and Celia said: 'When he left, he was given a gold cigarette case.
'He had that melted down and made into signet rings for all the grandchildren, engraved with their initials and birthdates. That's not a monster or an unfeeling man.'
Paxman's grievances have, in contrast, been a sad revelation.
'It's unfortunate Jeremy feels this way,' Celia says. 'He's written that he suffered from depression and we never knew. Keith would have wanted to help if he had known and he could have.
'He had insight and wisdom. I'll be interested to sit down and read the whole book, although I'm not sure if Jeremy will be sending me a copy this time. I'm just glad for Keith that he is not here to read it himself.'
