New Zealand market conditions have created the ideal low-budget package
In November, WhereScape founder Michael Whitehead flew to Boulder, Colorado to sit in a room and be quizzed for four hours by some of the top analysts in the field of business intelligence.
Getting an invite from Claudia Imhoff's Boulder BI Brains Trust is a sign of the impact the Auckland data warehousing firm is making in the arcane world of finding new and faster ways to look at data.
The month before, IBM had been in the same spot explaining its smart analytics system. In September it had been industry heavyweight Teradata, spelling out its thinking on the appliance market - a sort of data warehouse in a box that organisations can slot into their data centres.
"A week or so before us it was the BIS2 guys, Andrew Cardno from Wellington presenting, they've got fantastic visualisation tools, so it's good to see New Zealand software making waves there," Whitehead says.
Whitehead describes WhereScape's approach as almost Luddite: concerned with the basic building blocks of building an environment where an organisation's data can reveal things that may not have been obvious before.
"We got a positive reaction. The analysts at BBBT agreed this was not solved. Building a data warehouse was still a big problem, it still takes too long and once you have built something, it is too hard to change," Whitehead said.
Companies like WhereScape and BIS2 provoke interest because the big players in business intelligence are in consolidation mode as market conditions tighten, so innovation is pushed to the margins.
In recent times, IBM has bought Cognos, Data Mirror and SPSS, Oracle has bought Hyperion, Golden Gate and Sunopsis, its major rival SAP snapped up OutlookSoft and Business Objects, and even Microsoft has been pushing up into enterprise computing, taking onProClarity, Stratature and DATAllegro, which it is rejigging to run its SQL Server database in a massively parallel environment.
Teradata, whose purpose-built boxes have bucked the trend in computing to run everything on standardised servers, is going from strength to strength.
Teradata's systems are the ones that tell a retailer like The Warehouse how to arrange its stock, or an airline whether a flight is profitable before the plane leaves the tarmac, rather than knowing at the end of the month.
Whitehead say while large, complex enterprises with multiple terabytes of data still want the large systems, small and mid-market firms are also looking for data warehouses that won't bust their budget.
That's where solutions developed for New Zealand conditions can find acceptance worldwide.
"The New Zealand market wants to do it with a lot less resource, put less time into it, and has greater expectations," Whitehead says.
New Zealand also doesn't have the scale issues, so developers here don't get bogged down in scale issues - but if they can scale up, the world is their oyster. WhereScape's 150 United States customers include fast food chain Subway. Its software analyses what is bought with the 26 million issued Subcard loyalty cards.
"What you have in the US is the sheer volume of people using it every day, so the amount of tuna, the amount of tomato consumed - data volume is a problem. "In New Zealand, volume is not a problem so people want to use the system for more sophisticated analysis."
That means in New Zealand people expect to immediately start doing ad hoc queries of a data warehouse. In the US, analytics is a whole other project.
"We have a different way of attacking the problem. Generally, the competing toolkits are set up to handle large complex problems, which assume you have a well-resourced project with enough people and time.
"Then you have another section of vendors which address a relatively easy problem in a resource-constrained environment, and they're often a cube with Excel.
"The space we are in is where there are resource constraints but a complex problem. So when we go into the big wide world, people say, 'You are allowing us to have constraints and still get answers'. The US market's up 200 per cent for us this year," Whitehead says.
He says while there is a lot of talk about business intelligence, making it part of the day-to-day way organisations function, "a lot of major organisations still struggle with basic reporting on their information".
Changes anywhere in the system can change the way data is treated. A merger or new billing program can mean the concept of "customer" can suddenly change, and executives struggle to grasp what the data is telling them.
"Data warehousing can be a good way to solve basic business problems. Often the best way to solve problems is to grab the data, mash it up, and see if it is all coming from the same place."
Many organisations treat data warehousing as a project, often without thinking through the underlying architecture they need. Whitehead has turned that to WhereScape's advantage, providing tools so firms can fail quickly and move on quickly to building something that works.
"Because we have a product that can be built fast, it is often used for prototyping. We have companies that will build a prototype in our software, use it to generate our documentation, then send it to their outsourced development shop to build it in a traditional product."
