What are you doing to support the All Whites? Share your photos and videos here.
The All Whites must beat top of the table Paraguay next up, if they are to guarantee a spot in the next round.
Paraguay are without their top player Salvador Cabanas who has a bullet fragment lodged in his skull after a shooting in a Mexico City bar earlier this year.
But the South American side, who beat Brazil 2-0 in a World Cup warm-up, have proven they are dangerous even without Cabanas.
Paraguay seem to be using Cabanas' shooting as a point of inspiration. Paraguay President Fernando Lugo recently sent a letter to the team's players asking them to play for the former striker.
"Remember our beloved, stupendous example of gritty Paraguay spirit - Salvador Cabanas," Lugo wrote. "He is a magnificent example of how the fighting spirit, a strong will and faith helps us respond in the face of adversity."
And they did just that when they drew 1-1 with Italy and beat Slovakia 2-0. New Zealand drew with both sides 1-1.
Paraguay came out fighting against Slovakia with three strikers, including Manchester City forward Roque Santa Cruz.
Coach Gerardo Martino said the tactic allowed Paraguay to run the match.
"I think these results first have strengthened us mentally, spiritually not just as football players but in our attitude," Martino said.
The All Whites have already received a warning from Enrique Vera. Vera opened the account for Paraguay against Slovakia and was awarded man of the match for his troubles.
"We know we're getting closer but we know also that we must go on working and our intention is to be first in the group," he said.
Paraguay has a decent World Cup record - compared to New Zealand. They reached the second round in 1998 and 2002, but were knocked out in the group stage four years ago in Germany.
New Zealand, by contrast, have only been to the World Cup once before and that ended with successive losses and an early trip home in 1982.
- NZHERALD STAFF