To replace him at the wrong time and with the wrong person would trigger a state of resistance and infighting that would render the move counter-productive.
If Bridges is to be challenged or forced to resign, there has to be a trigger that almost everyone can agree justified the change - and it has not reached that point.
"Change the leader in case it gets dire" does not have the logic as a leadership change forced by actually being in a dire state. It is not in a dire state.
"Change the leader because Labour's pollster shows National below 40 per cent" is not compelling either.
The poll most likely to tip the balance against Bridges is Colmar Brunton – and its next one is expected this weekend.
The pollster's party vote often rates National higher than other polls so a sustained fail in that poll is still likely to be the most compelling trigger, in the absence of a catastrophic blunder by Bridges.
The party has definitely decided not to love Bridges – despite some of his own success. Success is not a word you often find in the same sentence as Simon Bridges but he has had a few.
The failures are always sheeted home to him. But the successes that Bridges has had through the Government's honeymoon phase are often given to other MPs or to John Key, Bill English and Steven Joyce.
Their most valuable legacy was their skill at picking messages and executing them effectively.
Bridges learned at their feet and he and his team have been very good at picking issues and creating messages that resonate with voters and get under the Government's skin.
It began with the scores of reviews, commissions and working parties set up by Jacinda Ardern and continues over cost of living issues, roading, and taxation.
As time progresses and official statistics show increases in state housing waiting lists, rents, numbers on job seeker support, hardship grants, hospital waiting lists, cancer treatment times, the scope for National is expanding daily.
Bridges has been especially successful on taxation. He capitalised on the vacuum between the release of the Tax Working Group report and the Government's decision to ditch any prospect of a CGT into the future.
And he is making the proposed "tax" on some gas guzzling cars to subsidise cleaner ones his own issue.
Bridges nudged his way into the Blair Vining petition for a cancer strategy at Parliament this week, as well he should. He needs to be associated with human stories that highlight failures of Government.
The health story, be it the underfunding of Pharmac, the treatment of cancer, the ambiguity over health targets, dropping immunisation rates, is not improving for the Government.
The large increases in health expenditure over two years appear not to be met by improvements in services.
And despite mental health getting $1.9 billion in the Budget, the Ministry of Health is now running "roadshows" to see where the money should be spent.
National has the Government under almost constant pressure.
Bridges is performing better in the media.
Bridges had an excellent week this past week. He was an even match for Jacinda Ardern in the House over the cost of living, which is not an unimportant arena, and he was made the subject of a Green Party attack ad over the car emissions policy.
There is no greater success for an Opposition leader than to be attacked for one's policy by one's enemies. He looked like the Cheshire Cat when being asked if he taken offence.
No doubt that his biggest disappointment was that the ad was pulled down so quickly.
It fairly accurately mimicked his pronounced Kiwi drawl but was interpreted as an attack on his accent and the Greens panicked after an hour and 40 minutes.
Bridges' accent is what defines him to many New Zealanders and it prevents them from even hearing his message.
It is not always there. In an interview this week, the accent and the robotic formality subconsciously disappeared the moment the video camera was off and it was just an interview with a digital recorder.
It is probable the Bridges drawl developed as a result of his courtroom work, where lawyers often slow down their speech to give themselves more time to compose their sentences.
It has stayed with him in politics but only in situations in which he speaks slowly. He could be trained out of it but he chooses not to.
It might make it harder for Bridges to seem prime ministerial, but no leader of the Opposition ever does.
Some of Bridges' caucus members have had their own successes, such as Paul Goldsmith against Shane Jones on the Provincial Growth Fund and Judith Collins in humiliating Phil Twyford over the woeful KiwiBuild numbers.
It is much more difficult to compete against a popular Prime Minister in Ardern who has emerged as a global figure of respect after the mosques massacre.
That is why National is seeking to "reframe" the contest.
In a contest about leadership between Bridges and Ardern, she will always win - especially with the obsessive attention given by some media on the preferred prime minister ratings.
The aim is to take the focus away from Bridges and to put the emphasis on "You," the voter - which will be making a virtue out of necessity.
Such is the lack of emphasis on Bridges this weekend that his keynote speech to the conference on Sunday will be early in the morning.
It will not be built up as the climax just before the conference ends as has traditionally been the case.
There will be no heightened expectation on which he fails to deliver. It is a sign of the uncertain state of his leadership.
He has an opportunity this weekend to make it more certain, one way or another.