Their confidence will be boosted further should they complete a rare clean sweep of the ODI series in South Africa in the final match on Saturday morning (NZT).
New Zealand skipper Brendon McCullum lauded the performance of his side, who produced their victory on the back of a masterful 145 not out from Kane Williamson and some dynamic fielding.
"I think very few teams ever play a perfect game so I think it's important to recognise that games and scores are often made up of a couple of individuals performing well and a few guys chipping in around them," McCullum said.
"So I think that's what we saw today - an amazing innings from Kane - but I think the partnership that him and Grant [Elliott] were able to manufacture really allowed us to give us a good launching pad and a couple of us chipped in towards the end as well. So there are some areas to work on, definitely, but let's not underplay what we've just achieved as well.
"I think it's a fantastic effort and, yes, we want to keep improving as a team, but we are pretty pleased with what we've achieved so far."
The recalled Elliott, who seemed about the sixth cab off the rank to even get a game before a host of injuries struck the squad during the past month, contributed a vital 48 as he and Williamson added 127 for the third wicket.
Their stand and a few late boundaries from Nathan McCullum (19) and Kyle Mills (15 not out) ensured New Zealand made it through to 279-8.
South Africa's top order were cruising at 167-1 before James Franklin and Brendon McCullum combined to run out opener Graeme Smith for 66.
That dismissal began a dramatic slide for the home side who lost wickets at regular intervals as the Kiwis fielded like demons and produced a world-record-equalling five run-outs.
Eventually, young all-rounder Jimmy Neesham delivered the knockout punch when he bowled Lonwabo Tsotsobe to wrap up the game and series and the New Zealand players ran from all corners of the ground to embrace each other.
While the win has given the side some much-needed credibility after their woeful test performances it has glossed over some alarming form slumps by senior players.
But for now, at least, it's time to celebrate and the New Zealand cricket side hasn't done much of that recently.
Read more: Winning score comes despite cramps