Just when things were looking promising, Yadav had Guptill lbw for 27 off 21 balls. Guptill stepped across in front of his stumps and was plumb in front.
The run rate was sharp as Williamson got into his work, producing a couple of fluent cover drives, having got off the mark second ball with an effortless straight drive off Yadav.
Williamson, coming off a century in the second ODI in Delhi, looked in fine touch, only to fall to the innocuous offspinning allrounder Kedar Yadhav.
The New Zealand captain tried to sweep, and missed it, leaving New Zealand 80 for two.
Latham, following 79 and 46 in the first two games of the series, went to his sixth ODI half century, in 59 balls.
Ross Taylor's form has been ropey on tour and he struggled again early with his touch.
However it eventually came to him, as he sent a rasping cover drive off left arm spinner Axar Patel to the cover fence, then slapped Patel for six to long on.
However on 44 he was deceived by a lovely ball from legspinner Amit Mishra and stumped. It ended a 73-run stand with Latham but from 153 for two, New Zealand went into another batting spin.
They lost six for 46 in 9.3 overs - four for 16 in 21 balls at one point - completely undermining the work of the batsmen earlier.
Corey Anderson sent a knee high full toss from Yadhav to deep mid off; Luke Ronchi was brilliantly stumped by MS Dhoni for 1, Latham lamely drove to cover, Santner got a leading edge to be caught at point and Southee had a heave and dragged a ball onto his stumps.
It made depressing viewing as it happened in good batting conditions, perhaps the best of the tour.
Neesham, sweeping and driving impressively, had strong support from Henry, who at one point took 14 balls in three successive Yadav deliveries in the 49th over.
The stand ended in that penultimate over when Neesham skied a catch to deep cover. It suggested New Zealand have hidden strength with eight wickets down. At Dharamsala in the opening ODI, Latham and Southee put on 71 for the ninth wicket.
Henry finished unbeaten on 39 off 37 balls.
It was a good day for the spinners, Jadhav taking three for 29 and Mishra two for 46, while the pace attack leader Yadav took a pricy three for 75 and Jasprit Bumrah finished with two for 52.