"In my view part of the issue for rugby is not unrelated to the lack of women in power, governance and management. And I don't just mean at NZR.
"The provincial unions elect the board. Go and look at how many women are there. The women's game is actually the fastest growing game.
"If you don't have women in the room...a lot of the violence is against women. We would get a different perspective if we had more gender balance on boards and in the game. There are some incredibly qualified women to step forward and do that."
Rutherford, the New Zealand rugby chief executive from 1999 to 2002, understood NZR was serious about making changes.
"But real change has to happen," said Rutherford, the chief commissioner for the past five years.
"All the major sports are committed to a whole lot of issues around integrity and inclusion, homophobia, gender discrimination, racial discrimination, violence and abuse.
"Sport around the world is in a great place to take leadership on this. I don't think any of the sports are hiding from their responsibilities.
"Some are ahead of others. Certainly the Warriors - partly because they are in the NRL which has been ahead - have taken issues of violence much more seriously."
He was cautious in commenting on the Filipo situation, saying the case was still before the courts because there might be an appeal.
Rutherford echoed Prime Minister John Key, saying that "victims should not be alone" and should be able to live without fear.
Rutherford said: "Victims should be front and centre...whenever they (rugby bosses) did find out they needed to go to the victims."