Being verbally abused, spat at and occasionally physically assaulted are things that happen too often they say.
"One worker asked not to be assigned to work with someone she didn't get along with," said Ronnie Welby.
"She was told that she had to work at that facility and she ended up having to lock herself in the office for her own safety."
A worker said staff are expected to treat their clients with respect at all times but that doesn't cut both ways.
"We can't be expected to always give respect when we are not receiving it."
Another said that new procedures have been implemented at some facilities without proper consultation and staffing levels are often unsafe.
The union sought mediation with IHC management but regional organiser Lyn Williams said they failed to come to the table.
"We will have to see what we're going to do but it looks like we will have to be out again and probably for longer next time."
The workers said managers were covering their shifts at Idea Services facilities during the four-hour strike.
Idea Services' chief operating officer Joan Cowan said management had "put careful plans in place to ensure the safety of people with intellectual disabilities during strike action".
She said Idea Services has been in negotiations with the union representing staff and remains committed to trying to achieve a collective agreement.
"We have listened to our staff," she said.
"We have been carefully considering all demands, either financial or otherwise, and are continually balancing staff remuneration with other quality improvement priorities."
Cowan said Idea Services is operating in a sector that is currently under-funded by $150 million a year.
However, E tū industry co-ordinator Alastair Duncan said IHC operates the same business model as for-profit care providers, spinning off its financially successful property division from its operational arm.
"IHC is a major landlord and property company that depends on its care staff. It is tragic to see them ignoring their own workforce."
Idea Services workers are asking supporters to call their management on 0800 422 422 and tell them to "be fair to those who care".