With a new swinging ball in hand Tim Southee completed his third career five-wicket bag and Trent Boult picked up three wickets in the session as the Black Caps needed just eight overs to wrap up the Sri Lankan innings and hold a 168 run lead.
The 168 innings lead is the second biggest by a New Zealand side in Sri Lanka and puts them in position to end a five test losing streak and claim their first victory in the whites since beating Zimbabwe in Napier 10 months ago. The Black Caps are also out to end an even bigger drought in Sri Lanka where they haven't beaten the hosts since 1998.
Southee finished with 5-62 and Boult 4-42, taking 3-8 from his four overs on day four.
The bad batch of balls continued in New Zealand's second innings with two replacements needed inside seven overs and led to the downfall of Guptill.
Guptill and Brendon McCullum got through to 32 before Shaminda Eranga had Guptill edging to Dilshan at gully for 11. It ended a forgettable time at the crease for the New Zealand opener in Sri Lanka having failed to pass 14 in all four test innings.
McCullum got to 35 from 43 balls and looked in ominous form with seven boundaries before he out stretched for a Herath delivery and was stumped by Prasanna Jayawardene.
With first innings centurians Kane Williamson and Ross Taylor at the crease the Black Caps were in good stead to build a commanding lead before the wickets started to fall. And fast, with three departing in four balls.
Williamson (18) produced a tentative drive to a wide delivery from Nuwan Kulasekara which sent an edge to first slip. Kulasekara then captured Daniel Flynn with a first ball duck LBW and New Zealand were 74-4. Kruger van Wyk joined Flynn with a golden duck, caught at leg slip to Herath to complete a four-ball pair for the match.
At 75-5, skipper Taylor and newcomer Astle steadied the ship, adding 79 for the sixth wicket, and ensured New Zealand remained in control of the match.
SCOREBOARD
New Zealand
First innings 412
Second innings
M Guptill c Dilshan b Eranga 11
B McCullum st Jayawardene b Herath 35
K Williamson c Paranavitana b Kulasekara 18
R Taylor not out 44
D Flynn lbw b Kulasekara 0
K van Wyk c Paranavitana b Herath 0
T Astle not out 27
Extras (lb 0, nb 1) 1
Total (for 5 wickets, 45.0 overs) 140
Fall: 1-32 (Guptill), 2-56 (McCullum), 3-74 (Williamson), 4-74 (Flynn) 5-75 (van Wyk)
Bowling: N Kulasekara 12-2-47-2, S Eranga 10-1-39-1 (1nb), R Herath 17-3-42-2, S Randiv 8-1-23-0 (1nb)
Sri Lanka
First innings (overnight: 225-6)
T Paranavitana c van Wyk b Southee 40
T Dilshan b Southee 5
K Sangakkara c Boult b Southee 0
M Jayawardene c Williamson b Boult 4
A Mathews c Guptill b Southee 47
T Samaraweera c Guptill b Boult 76
P Jayawardene c Williamson b Patel 12
S Randiv LBW Boult 34
N Kulasekara c Taylor b Southee
R Herath c Williamson b Boult 5
S Eranga not out 3
Extras (3lb, 1w, 3nb) 7
Total (all out, 94 overs) 244
Fall: 1-7 (TM Dilshan,), 2-7 (Sangakkara), 3-12 (Jayawardene), 4-102 (Paranavitana), 5-103 (Mathews), 6-128 (Jayawardene), 7-225 (Samaraweera), 8-232 (Randiv), 9-240 (Kulasekara)
Bowling: T Southee 22-4-62-5 (1nb), T Boult 21-7-42-4 (1w, 2nb), J Patel 22-3-47-1, T Astle 13-2-41-0, D Bracewell 13-1-44-0, K Williamson 3-1-5-0
New Zealand lead by 322 runs on the second innings with five wickets in hand