Video from the chamber shows the father of three rocking the baby as he warns an MP his speaking time has expired.
It's not the first time an MP has brought a baby to Parliament - in 2017 Labour's Willow-Jean Prime brought her 3-month-old baby Heeni with her while the House debated extending paid parental leave.
Many people on social media praised the Speaker's actions.
Other MP's were delighted at having the baby at Parliament.
Green party MP Gareth Hughes shared a photo saying: "Lovely to have a baby in the House, and what a beautiful one."
Green MP Golriz Ghahraman tweeted: "Who needs to see this today? Every single last one of us, that's who."
This was in stark comparison to Denmark's Speaker's reaction when another legislator brought a baby into Parliament a few months ago, SBS News reports.
"You are not welcome with your baby in the Parliament's chamber," the Speaker said, kicking out the legislator.
Also a week ago, a video of Zuleikha Hassan, a Kenyan lawmaker, circulated after she was kicked out of parliament for bringing her baby to work.
When Mallard became Speaker in 2017, he promised to make Parliament a more family-friendly place.
After New Zealand Prime Minister Jacinda Ardern brought her baby daughter - Neve - to her debut speech at the United Nations in New York, politicians are increasingly following her lead and proving that parenting and politics can go hand-in-hand.
