Gordon said hearing of Lilly's death was as if a heavy weight had finally been lifted off of him after all these years.
Lilly's own daughter, Samantha Sloane, said despite losing her father, she understood what this meant for the Gordon family.
"As hard as it is for me, that was one of my very first thoughts and one of the first things I said to my mother: 'I actually feel a sense of relief for the family, because I know they'll be relieved'."
The 26-year-old was three years old when her father committed the crime that would take away another little girl's mother, something she felt particularly guilty about, she said.
"I told my Dad that what he'd done is absolutely disgusting and it's unforgivable. My heart's always broke for [Linda's] daughter because I got my Dad still and she didn't know what happened to her mother and that's really sad.
"I want to thank them because they were so good to my mother back then. Even though I didn't do anything wrong, I am really sorry for their loss and I actually do feel a sense of relief for them because I know if that were me, I would feel the same.
"I want to let them know that he did hate himself for what he'd done and he'd changed a lot."
Lilly was sentenced to life imprisonment with 15 years' minimum non-parole and released from and recalled to prison twice before being released for a third time in November 2015.
He is one of 19 people who have died on New Zealand's roads during the holiday road toll, which finishes at 6am tomorrow.
It is the highest toll since the 2011/2012 period when 19 people also died.
At the time of her death, Linda Gordon had a 2-year-old daughter, Laura. Gordon and his wife raised Laura, who now has two children of her own.
Gordon phoned his granddaughter about the news shortly after receiving the call.
"She's always been terrified he's going to turn up on her doorstep. And I must admit, I was sort of thinking one day he'd turn up on ours.
"It all comes back to you all the time. I must admit, it's the biggest relief I've ever had.
"Christmas has always been thinking of Linda because it was Christmas Eve day before we could get her back for her funeral."
Linda Gordon's death had taken a huge toll on his wife, who died of cancer several years ago.
"She tried to put the whole thing out of her mind. She wouldn't discuss it, wouldn't talk about it and refused to accept the fact that Linda had actually got killed."
Gordon said he did feel for Lilly's mother.
"It's got nothing to do with her. We've got nothing against her."