The pair combined for a 96-run opening partnership.
Latham's dismissal brought Williamson to the crease, who again looked composed.
Guptill continued to score freely at the other end but eventually fell to Hardick Pandya.
Ross Taylor came in and struggled for fluency in his 58-ball innings, scoring 35 runs.
The fall of Williamson at 184-3 was quickly followed by Jimmy Neesham eight runs later, bringing BJ Watling to the crease in his first ODI since 2013.
Watling was caught on 14 and once Taylor was run out, New Zealand's good platform was in the rear view.
Short cameos from Anton Devcich, Mitchell Santner and Southee saw New Zealand through to 260-7, a solid total to defend.
The Black Caps started out strongly as Southee had Rohit Sharma caught behind for 11.
But India settled through Ajinkya Rahane (57) and third ODI hero Virat Kohli (45), combining for 79 runs.
Recalled spinner Ish Sodhi provided the big breakthrough, forcing an edge from Kohli.
Rahane was the next to go when he was trapped in front by Neesham and when he uprooted Dhoni's leg stump, India were in a spot of bother at 135-4.
Williamson then decided to reintroduce Southee to the attack, which proved to be a master stroke.
Southee took the wicket of Manish Pandey through an excellent catch from Latham and provided an even better delivery next ball.
Kedar Jadhav took the crease but was no match to a pinpoint slower ball from Southee.
A couple of wickets later the game looked won with India at 207-9, but tailenders Dhawal Kulkarni and Umesh Yadav stuck around for a while.
Taylor ended the resistance with a sharp catch off the bowling of Trent Boult, giving the Black Caps the win.
The series decider is tomorrow at Visakhapatnam.