They agreed the promised inquiry into state abuse had potential to address some of those problems - but only if it was done right.
"For me a key issue is around historic abuse and understanding what happened," Gibson said. "Lobotomies, LSD experimentation, physical and sexual abuse - disabled people suffered a range of things that that no one should be expected to go through.
"The inquiry - they have talked about children in state care but it needs to include adults as well."
Martin, who grew up in foster homes and spent time at the Kimberley Centre, said he saw abuse and experienced it himself first-hand.
He described watching a child with an intellectual disability being hosed down by a staff member with a fire hose, and being warned "you'll get the same if you misbehave".
"The sad thing is that people are dying off. Where I'm from, there's only three left, and there used to be over 50 of us."
Gibson said, regardless, it was important for people like Martin to have input into any inquiry.
However he was concerned, particularly given the new Government's Speech from the Throne didn't mention disability at all.
"That doesn't resonate well with many of us," he said.
The Government has promised that within its first 100 days in office it will launch an inquiry into the abuse of children in state care.
Minister for Children Tracey Martin has said she is talking to officials about the shape of the inquiry, and it will have input from at least 20 organisations.
At least 100,000 New Zealand children and disabled adults were taken off their families and held in state institutions between 1960 and 1990. Successive reports have argued abuse was systemic during that time.
The previous Government refused to hold an inquiry into the issue, despite a petition supported by the Human Rights Commission with 12,000 signatures, and a report from the United Nations Committee on the Elimination of Racial Discrimination released in August calling for an apology.
Since the new Labour Government was elected, a range of interest groups have expressed concerns around what will be included - with some saying it should be limited only to those in state care, while others say it should include groups as diverse as churches and sports clubs.
New Disability Rights Commissioner Paula Tesoreiro assured the conference in her speech that the abuse inquiry was on her agenda, and said she and the Race Relations Commissioner Susan Devoy would both advocate that disabled adults be included.