At the moment, Louise Upston is the incumbent, but she has always given me the impression that she's uncomfortable with the role. Unsure of her role, perhaps.
In any case, she's been ineffective.
The writing was on the wall when she said she wasn't a feminist.
She said she wasn't going to be a flag-waver for women's issues, which, well, what is a Women's Affairs Minister for?
Paula Bennett and Judith Collins happily shoulder being feminists.
And that's because if you believe men and women should have the same rights, then you're a feminist. It's pretty straight-forward.
In fact, I've said on this programme before that my brother, Jason, who's a pretty raw farmer sort of a bloke, has a daughter and I've said he's a feminist because he believes his daughter should have the same rights as my son.
He's feminist. A feminist in Red Bands.
But back to the Women's Affairs Minister.
Louise Upston has been ineffective. Plain and simple.
She said she's not a flag-waver for women's issues.
She wouldn't comment on Ponytailgate - when the PM pulled a waitress' ponytail. Sure, it's difficult ground to cover when the boss does something wrong, but you still need to comment.
Upston voted against same-sex marriage.
She once said beauty pageants could give a young woman confidence.
She wouldn't comment on the Chiefs stripper scandal when the New Zealand Rugby Union failed quite spectacularly with its flawed internal investigation.
Collins stepped in on that issue and said that investigation didn't feel right, and she ''was'' right.
Bennett has taken on big issues as well. It was Bennett who took the bull by the horns and took on Wicked Campers and those appalling, appalling slogans emblazoned across the side of their campers.
Upston, again, silent. She was the Minister of No Comment.
And yet we need a Women's Minister who will be more visual and vocal on issues such as family violence, pay equality, diversity on boards, paid parental leave... the list goes on.
So English, I think, has to replace Upston.
Who with? Could Bennett shoulder that role as well as her other portfolios? Not sure. That's a heavy workload.
But Crusher could, I reckon.
Collins as Women's Affairs Minister.
She would front-foot every issue she needed to because, well, Crusher front-foots everything doesn't she?
The Women's Affairs role sits outside Cabinet, and that's another issue too, but I hope that while English is reshuffling his Cabinet, he also appoints a bold, effective Minister of Women's Affairs.