Lakes District Health Board has launched an investigation has begun and said it "sincerely regrets" the incident.
Both parents said the todder's thumb was normal before the insertion of the needle. They said staff did not give any explanation of what had happened, just saying it should return to normal.
The test results came back negative for meningitis and Benedict John was discharged on Wednesday.
Mr Varghese said his son's thumb was still bent and a nurse, who was not present when the needle was inserted, asked them who had put the IV in. She told them it should get better after a couple of hours.
The couple took Benedict John home but a few hours later the thumb had not improved and seemed painful to the touch.
"When I touched it [his thumb] he would cry out 'no no no'," Mr Varghese said.
They returned to the emergency department where staff took X-rays and spent about half an hour trying to manipulate the thumb back into place, applying cream then an injection to numb the pain.
Mr Varghese said doctors eventually sent them away, telling them to wait a few days to see if the problem resolved itself. It didn't, so they went back to the emergency department on Easter Monday and asked to see a specialist. The specialist informed them it required surgery, which is scheduled for today.
Throughout the visits, no explanation was given for how the thumb was dislocated.
"They told me normally it's not happening but, unfortunately, this happened and it's not good," Mr Varghese said.
He said staff admitted the injury happened while the needle was being inserted and gave him a complaint form to complete.
The couple lodged a complaint and had since been contacted by an investigations officer.
"She told me sorry," Ms Antony said. "I told her 'it's happened, you sort out the problem'."
Although the couple would prefer their healthy child not to have surgery, they realise it's necessary to prevent future problems.
"It's his right hand, when he goes to school he will need to write," Ms Antony said.
Although hospital staff apologised, Mr Varghese said they were considering getting legal advice.
"It's a baby, eh? If it was my finger I wouldn't worry about that," he said.
Lakes District Health Board communications officer Sue Wilkie said the board sincerely regretted the incident and any distress caused to the family.
She said a senior staff member had contacted Ms Antony last Thursday morning to acknowledge the parents' concern and confirm that an investigation would take place but would be delayed slightly.
"The investigation is still ongoing, with a number of people needing to be contacted," she said. "It is hoped to have the investigation completed by the end of the week, at which time Lakes District Health Board will again contact the parents to arrange a meeting."
