Children in Clevedon make a snowman during a nationwide snowstorm in July 1939. Photo / NZ Herald
Children in Clevedon make a snowman during a nationwide snowstorm in July 1939. Photo / NZ Herald
A world-first project could dramatically improve what we know about New Zealand's weather, by using artificial intelligence to trawl through handwritten records stretching back more than 150 years.
The initiative will see Microsoft partner with Niwa to effectively train handwriting recognition technology to read historic weather logs, so they can be loaded into a database.
For scientists like Niwa's Dr Drew Lorrey, rescuing these old records had been painstaking but crucially important, as all the data they capture is used in supercomputer-driven models of past daily weather.
Those old logs include a range of archival material that could yield exciting insights about our weather history.
The first step in the new project is to use weather information recorded during a week in July 1939 when it snowed all over New Zealand – even at Cape Reinga.
"Was 1939 the last gasp of conditions that were more common during the Little Ice Age, which ended in the 1800s - or the first glimpse of the extremes of climate change thanks to the Industrial Revolution?," Lorrey said.
Weather records at that time were meticulously kept in logbooks with entries made several times a day, recording information such as temperature, barometric pressure and wind direction.
Snow covers the hills around Clevedon, near Auckland, in July 1939. An AI-driven project could help scientists answer why strange weather events like this happened. Photo / NZ Herald
Comments often included cloud cover, snow drifts or rainfall.
"These logs are like time machines, and we're now using their legacy to help ours," Lorrey said.
"We've had snow in Northland in the recent past, but having more detail from further back in history helps us characterise these extreme weather events better within the long-term trends.
"Are they a one-in-80-year event, do they just occur at random, can we expect to see these happening with more frequency, and why, in a warming climate, did we get snow in Northland?"
Until now, however, computers haven't caught up with humans when it comes to deciphering handwriting.
More than a million photographed weather observations from old logbooks are currently being painstakingly entered by an army of volunteer "citizen scientists" and loaded by hand into the Southern Weather Discovery website.
This is part of the global Atmospheric Circulation Reconstructions over the Earth (ACRE) initiative, which aims to produce better daily global weather animations and place historic weather events into a longer-term context.
In New Zealand, scientists were already seeing benefits of obtaining old observations for improving understanding and long-term record of the Southern Annular Mode, which affects westerly windflow and New Zealand's rainfall.
"These logs are like time machines, and we're now using their legacy to help ours," Niwa scientist Dr Drew Lorrey says. Photo / Dave Allen, Niwa
"Some of the data have important historic and cultural value too, because they are part of our first instrumental observations taken in this part of the world."
As Microsoft NZ's senior cloud and AI business group lead Patrick Quesnel put it: "Old data is the new data."
Technology was finding better ways to preserve and digitise old data reaching back centuries, which in turn could help us with the future, he said.
"We hope this project will bring inanimate world weather data to life in a way everyone can understand, something that's more vital than ever in an age of such climate uncertainty," he said.
"And the wider applications of using AI for handwriting recognition go far beyond that, with the ability to preserve so much more scientific and cultural data from old archives that might otherwise lie unread, or worse, be lost forever."
He noted that Microsoft already happened to be applying its cognitive search functions to material such as the so-called JFK Files from US President John F Kennedy's 1963 assassination, hoping to learn more about the past by cross-referencing millions of old documents.
"You could apply the same technology to legal documents, to provide legal precedents, or analyse diaries and letters from around the world and potentially discover lost artworks mentioned somewhere in the records," Quesnel said.
"In reality, you're limited only by your own mindset and understanding of how AI and machine learning can be applied."
