Sri Lanka were finally dismissed for 195 early in the final session, handing the Black Caps a 167-run victory.
'It was tough after losing so comprehensively in Galle,' captain and man-of-the-match Ross Taylor said, "but the way the guys stepped up was a credit to them. We will treasure this moment for a while.
"Test wins away from home don't happen very often."
It hadn't happened for nearly 12 months, when New Zealand tipped over Australia in Hobart. It was also Mike Hesson's first test victory as coach, New Zealand's fourth test victory in Sri Lanka and their first there since 1998.
It was appropriate Trent Boult had the final say when he had Mathews caught at slip after an impressive series.
Boult (3-33) and Tim Southee (3-58) combined for 15 wickets in the match and were menaces throughout with their ability to swing the ball. They put constant pressure on one of the world's best batting lineups and provide some hope the Black Caps can climb out of the dreadful hole they have been in.
One test win doesn't make up for what came before but they at least showed they can fight - a quality not seen enough this year - and showed the value of numerous contributions.
Too often in 2012 players have attained success, albeit limited, in isolation. In Colombo, it has been a team effort. New Zealand has some good cricketers but few world-class players (some might argue there aren't any) and success can only be achieved if a number chip in.
Taylor and Kane Williamson with the bat and Southee and Boult with the ball were the obvious standouts from this Colombo test but others like Todd Astle and Doug Bracewell also did their bit, not to mentioned some of the catching.
It is the only way they will compete in South Africa. The Proteas, however, are not only the world's best side but also the ultimate fighters as they showed in this week's epic draw.
Starting the day at 47-4, Sri Lanka heaped more pressure on themselves when Thilan Samaraweera was needlessly run out for seven with the score at 63.
New Zealand might have felt they were on the verge of victory but they were frustrated as Mathews and Prasanna Jayawardene (29) dug in. The pair combined for 56 in 33.3 painstaking overs on the docile pitch.
Astle finally made the breakthrough when the leg spinner induced a faint edge from Jayawardene for his maiden test wicket and Boult and Southee did the rest.
There is still plenty of work to do until New Zealand can say they have turned things around but it's a start.