Yesterday, a forensic examination continued at the home in Carlton Cres while a post-mortem examination was carried out. Police said they would not be releasing the results.
Mr Matene told 3 News he felt "helpless" he was not there to help his son.
"I feel it hard," he said. "Why wasn't I there for him? Why I couldn't do anything? I was helpless."
His relationship with Ms Taylor ended a month ago and Atreyu was spending a couple of weeks at Ms Taylor's home, 3 News reported.
Mr Matene and his father, Dez Matene, had to identify the child's body yesterday.
"Jess was a good mother," said Dez Matene.
"She's a good mother, she loved her son, we loved her, we all loved each other. God had another calling for [Atreyu]."
Other family members went online to share their grief.
Claire Edmonds said she'd visited the Carlton Cres home recently.
"Im sooo trying to keep the peace for my nephs, but sticking together will get us through this, one day at a time. He [Atreyu] was just one month older than my Sarah."
Ms Taylor's sister Jane Walmsley wrote on Facebook: "My heart aches as I type this message, my little sister will be burying her baby boy in the next couple of days. No parent should ever have to bury their child."
Others called for calm.
Gary Gear wrote: "My condolences to you in this time. And as to all Facebookers that haven't heard the news. Please don't judge Jess at the moment until all the facts have been heard. Arohamai Jess stay strong and be with family OK."
Ms Taylor's father, Donald McGahey, said the funeral would be held on Wednesday.
A week ago today, the toddler celebrated his birthday.
Before Atreyu died, Mr McGahey asked for support from family and friends because the boy had "had a fall".
Twenty-five staff are working on the homicide inquiry.
