Staff visited six of the nine residences and found four were safe.
But concerns about Te Puna Wai o Tuhinapo in Christchurch, one of the four youth justice residences which has 30 spaces, led to a "detrimental elements" rating - the first time such a red flag had been raised.
Staff were struggling to manage young people's behaviour.
"Young people were acting out more and there were a relatively high number of incidents, with high use of staff restraints and secure care."
Restraint means a resident is safely held if they are a risk to themselves, others or property.
All youth justice residences and four out of five of the other residences have a secure care unit - a locked section where people are temporarily placed as a last resort if they become a danger to themselves or others.
A court must grant permission for a young person to be held in secure care for more than 72 hours.
An unnamed residence was found to have leadership issues and an "unsettled" environment.
Follow-ups confirmed progress has been made and both residences are "back on track".
While material conditions at residences are adequate, the report finds most have an institutional feel that is not "youth-friendly" or "home-like". Problems include broken air-conditioning units, windows and gates.
All residences visited were moving in the right direction, the report found, although from very different starting points. Two residences had excellent practices.
Last year, the National Union of Public Employees warned violence against staff at Te Puna Wai o Tuhinapo was nearing crisis, with a number having claims with ACC.
Chris Polaschek, general manager of youth justice support for CYF, said the initial visit to Te Puna Wai o Tuhinapo was in May last year and an action plan was immediately put in place.
Minister for Social Development Anne Tolley said the future of residences will be considered as part of an overhaul of care for young people.
"I'm not convinced that they are the best option for some of the young people placed in them, and that alternatives in the community might provide better long-term support, while keeping safety in mind."
Labour's children spokeswoman Jacinda Ardern said the findings showed how important regular visits by the Commissioner were, but that they had reduced to every 18 months because of resourcing constraints.
The report's key recommendation is to bolster efforts to move to a child-centred culture at CYF.
Ms Tolley said the Vulnerable Children's Board was involved in overseeing changes. "The culture change has already started ... we are not waiting until April next year to make improvements. A programme of continuous improvement includes making sure children's voices are heard, intensifying work with Maori, and more efficient management of case loads."