A full-scale search was started, with police and neighbours looking for the infant, who was found by his dad just 200m from the family home about 45 minutes after he went missing.
"It was nobody's fault, he was just so fast," Ms Tapp said.
"He's never spent a day without us in his life because of his special needs and he's never been out of our eyesight. I watched him go around that corner and I knew my husband was just there. So we're refusing to say, 'Could have, should have, would have'. He was gone so quick."
Ezra was a bright and happy child, who "had the sweetest, biggest heart", Ms Tapp said.
His autism meant he did not speak, but he would "run everywhere".
"He did not stop running from 6am to 10pm. And I can just imagine the moment he escaped that gate and managed to slip past his dad. I can just see it in his head, 'I've finally done it, I've finally escaped, I'm free'.
"He hated being boxed in. If there was a gate or a door or anything, he just had to get out of it."
Yesterday, the family noticed the front-door lock was jammed, and found pieces of toothpicks Ezra had been using to try to pick the lock.
Despite the tragedy, the family are seeking comfort in the news Ezra could save others, after they learned his heart was suitable to donate valves to two or three infants of a similar age with heart conditions.
"There's a real shortage of replacements for those heart valves and we just thought, 'We've lost our son, and we can stop these other families from losing theirs'," Ms Tapp said, describing it as "the greatest gift".
"We have to give those valves to those other families. We couldn't save our boy; we have to save theirs."
The local community have rallied around the Tapps. A Givealittle page has been set up, and tributes poured into social media sites to offer condolences to the young family.
Ms Tapp has children Zoe, 12, and Liam, 8, from a previous relationship.
Ezra was musician Mr Tapp's only child - a nice surprise after being told he would be infertile following prostate cancer some years ago.
Ezra's death was not being treated as suspicious, Senior Sergeant Peter van der Wetering said.
Only a small number of people die in circumstances that allow their organs to be donated. Last year, organs and tissues such as heart valves were given from 36 deceased donors for transplant, and tissues only from 57, according to Organ Donation New Zealand.
Can you help?
To donate to the Tapp family visit www.givealittle.co.nz/cause/EzraTapp
