It marked the 12th time in 23 tests that Guptill had passed 50, but he has only converted two of those half centuries in to tons and those three-figure scores came against Bangladesh and Zimbabwe respectively.
Brendon McCullum also failed to go on and post the major individual total his side needed when he tried to cut a ball that was too close to his body and he played a Kemar Roach delivery on to his stumps and he was dismissed for 84.
It was the sixth time in 65 tests that McCullum has been sent packing in the 80s and he hasn't scored a test century since November of 2010.
The 30-year-old looking in typically attacking touch as he took 18 from a Marlon Samuels over but if New Zealand wanted to give themselves a better chance of staying in the test, he needed to be at the crease at the close of day four.
Guptill's opening partner Daniel Flynn also got himself settled as he rapidly made his way to 20 before Narine trapped him leg before wicket.
New Zealand's hopes will hinge on Taylor's ability to bat for at least two sessions, while he will need support from the incoming Kane Williamson, Dean Brownlie and Daniel Vettori to save the test.
Earlier in the day, the West Indies were bowled out in their first innings for a whopping 522 as Narsingh Deonarine finished with 79 and captain Darren Sammy added a neat 50.
Veteran seamer Chris Martin finished with 3-134 but was expensive as he went at 4.46 runs per over from the innings, while Doug Bracewell and Kane Williamson each claimed two wickets.
Daniel Vettori's struggle to breakthrough the West Indies' batting line-up continued as he bowled 51 overs but only managed to pick up 1-124.
Kruger van Wyk was tidy behind the stumps as he bagged two catches, took part in a run out and didn't conceded a single bye in 163.3 overs.