Adam Gifford: Unravelling bureaucracy's tangled web

Much useful data is kept from the public's eyes. Photo / Northern Advocate

Ever wondered what happens to all that information the Government collects? Think you can make better use of it than the bureaucrats? Need some facts to give your mash-up some muscle?

A new initiative by a group of digital activists aims to identify sources of public information, classify who "owns" it, what licence it is distributed under and if it is free or not. Open government ninja Glen Barnes says the Open Data Catalogue is from open.org.nz's practical manifesto.

"We have paid for that information, and I believe we have a right to it," says Barnes, whose day job involves turning property information into useful applications.

Some information must be kept behind departmental walls to protect individuals' privacy but there is a lot more which can quite safely be let loose.

To make it easier for local bodies and central agencies to let their data out, Barnes is working on an API (application programming interface) for data which is not available in easily digestible formats like Excel or CSV (comma separated values), such as information from websites written in HTML.

He'll take it along for discussion next month at the first open government data bar camp, a user-generated conference to be held at the National Library in Wellington on the weekend of August 29.

"I'm also taking down some work I'm doing on real-time transport information - some of the councils are interested in the concept of how things can happen from that.

"Local bodies often do not have the resources to build websites, but they might make data available for private enterprise to do it."

A typical example is mashing up Google Maps and crime statistics, giving people a visual impression of risk in their town.

Open data scares some agencies and politicians, as evidenced by Education Minister Anne Tolley's contortions when questioned about the league tables her national testing programme will inevitably generate.

It also raises questions about the way government agencies have treated data in the past.

Think of the Companies Office site, which is an excellent and free source of information, but was first designed with the aim of charging fees to offset its development and running costs. It's burdened with some clunky APIs - so for example it's not possible to search which files have been recently updated.

Laurence Millar spent the past five years as government chief information officer trying to streamline the Government's information systems and get better outcomes from its $1.9 billion IT spend.

In his last blog posting as a public servant, Millar wrote of a need to recognise the network effects of opening up government data in a form that means others can access it.

"Economic value is created by businesses building innovative new services using government data.

"Public value is created by enabling a richer and deeper understanding and dialogue among interested individuals about what the data tells us about our lives."

Millar says the new information culture will need government agencies with the inclination to put data out, communities who want to do work with the information and the tools to make it interpretable, such as the graphs available on gapminder.org.

Changes in the way the Government treats its information needs to come from the top, with political leaders who understand technology - a commodity in short supply here. Millar reported to Trevor Mallard, Annette King, David Parker and Tony Ryall.

"It's not a natural domain for politicians. If you look at countries which have made progress with IT in government, they are ones with a leadership which says it is important.

"In the United States, the Obama administration does get it. Northern European countries who put a premium on innovation and engineering get it.

"In Singapore and Korea there are people in power who understand information, network economics and externalities and growing the size of the pie. While we are slugging it out here in terms of domestic competition, the rest of the world is sailing away to different a future."

In his five years trying to encourage whole of government systems "we made good progress but not stellar progress".

The challenge was getting collective action out of a system where each government agency is required to be selfish.

"They have to look at what will be best for my ministry or my department and what we know is the difficult things we are trying to deal with as a country - healthy communities, safer communities, safer children, better business capability - can't be done with one department.

"By splitting things government does into individual components and trying to make each component as efficient as possible, you lose the larger outcomes."