"I was walking along the corridor and Parekura came up to me and he looked at me and he said 'Ahh, I see you're making up words'.
"And I said 'Yeah, and I didn't know I was.' He went: 'Me neither.' And he laughed to himself and wandered off."
Mr Key said he respected the MP as a passionate advocate for Maori and his constituents, and said he was loyal right down to his bones.
"Parekura was loyal, and that loyalty was tested on a number of occasions...loyal to his Prime Minister, loyal to his party, loyal to the people he represented and loyal to his country."
On Monday the house would mark the loss, Mr Key said. They'd remember a man who could fight and struggle for the issues he believed in, but do it in a way where he didn't make enemies.
"Not every parliamentarian could do that, but Parekura could.
"It's interesting thing that a guy that came into parliament and was loyal to one party and one party alone engendered emotions right across the parliament."
Earlier , Hone Harawira also had Hauiti Marae mourners in stitches as he recounted the best moments of his friendship with Mr Horomia.
The pair have had a long friendship stretching back 25 years - the former Maori affairs minister once employed the Mana Party leader.
Mr Horomia worked for the Department of Labour in his younger years.
Mr Harawira said they asked Mr Horomia to send home all the Pakeha staff so the Maori staff, who weren't happy with their boss, could clear the air over bubbling issues.
"He wasn't always nice," he said.
"We got into this big argument. Man you fullas think I'm a loudmouth ... you should have heard this fulla ... When it was all done we went for a feed. He was just that kind of guy."
The Maori Party, Tuhoe and Government Department staff who worked with Mr Horomia when he was a minister were among the groups to visit Hauiti Marae this morning for his tangi.
Maori Party co-leader Tariana Turia was weeping as she walked on to the Marae early this morning.
Her colleague Te Ururoa Flavell said afterwards that although Mr Horomia had led the Foreshore and Seabed Act that drove Mrs Turia to quit Labour and set up the Maori Party, Mr Horomia had also provided valuable counsel when the Maori Party was in its fledgling stages.
"Most people suspected there was a fair bit of tension but it was never like that with Parekura. He was one of the first people to come when we first arrived in Parliament and said 'look, I'm here to help with anything. He'd just come and sit down at the table.
"None of those barriers were put up outside the political forums. We might have had different views, but the great thing about him was that we just sat down together as Maori, discussed things as Maori. I suspect in my heart we all wanted exactly the same things."
"He was a wonderful man with a wonderful heart and as you can see from the people coming, well loved throughout the country."
