CF: It's important to businesses as it not only provides new opportunities to gain new insights into their customers behaviours and into their business performance, but also lowers the typical barriers to entry that have been historically associated with gaining insights from data such as cost and capability.
PP: How has big data made a difference to businesses?
CF: The concept of big data is still relatively new and uncharted, so it's still emerging but there are a few. For example;
* The now blueprint for recommendation engines at Amazon (customers who bought this also bought this)
* The famous Target example where they were able to detect that a teenage girl was pregnant due to her spending habits, to GE investing a billion dollars for a 1% gain in efficiency across their business
* A large travel company changing it's definition of loyalty from "repeat business" to "digital evangelists" based on their social media and web comments
* Healthcare organisations offering better care, more efficiently, based on better insights about their patient.
PP: Do many businesses get the whole big data thing yet?
CF: Not many no, like anything emerging, it's ripe for misunderstanding and inflated expectation. The challenge with this concept with data is that it's highly receptive to the "art of the possible" in that it doesn't take much for the imagination to run wild with the possibilities data can deliver. A lot of these possibilities are unrealistic or complex to achieve. That's why we think the first step is one of education, going back to the rule book and helping customers understand more about the data they have before getting too carried away with what's possible.
PP: It must have pretty big impacts on corporate IT requirements?
CF: Absolutely, that's why it has such a close marriage with Cloud Computing where companies can "rent" IT resources on a "pay-per-use" model. Requirements in the data space require a lot of power at irregular intervals of time, and ultimately are experimental in nature, so being able to align cost with this variable demand and to be able to fail with very little regretful spend is key to success without breaking the bank.
PP: You've previously spoken of data scientists as an emerging high value skill, how will that work?
CF: I'm a firm believer in a healthy balance of traditional analytics use cases such as data warehouse and business intelligence and the experimental side of data. Data Scientists operate in that experimental side, testing hypotheses, looking for correlations that answer questions the business is asking, usually using data from outside the organisation to find an edge.
PP: Do you have any examples of how big data can make life easy for customers?
CF: This is where the art of the possible starts to come in! There are literally thousands of examples, from improving transportation by amalgamating data from multiple sources to suggest the best route to take, to gene sequencing that looks for patterns of mutation in diseases for more effective treatment, to the more commercial use cases like helping you choose the best product to buy, or reminding you to grab your mobile phone charger because when you leave the house you would normally take it with you.
PP: Privacy and security must be huge issues though?
CF: Yes, it does tend to dominate most discussions on the topic, but it's also a huge opportunity as this area is still largely nascent in this new data context. That's why groups like the NZ Data Futures Forum are emerging because it's something that needs to be discussed. In my view, I don't think privacy legislation is necessarily the right path. I think a public opinion approach is more appropriate as it's constantly changing.
Ultimately, customers want to have the choice to share or restrict their personal information based on what value they're obtaining. It needs to be a continuously open dialogue. That's why I really like the concepts around dynamic privacy where customers can choose or set their privacy based on their preferences and change it if they want to.
In terms of security, it's a global issue seldom discussed. Bottom line is, there are more hackers in this world than security professionals to combat them. We need to work smarter. That's why I think we need to consolidate our information and protect it with the few security experts we have. The more dispersed it is, the more attack vectors hackers have to get information.
PP: Will the Internet where everything is connected impact on big data?
CF: Absolutely. Machines don't sleep! Machine generated data already exceeds human generated data, and as we create more machines that generate data that will increase even more. This will introduce both more opportunity as well as complexity in this space.
PP: Do you see any the future opportunities for New Zealand with big data?
CF: Definitely. I spoke at a recent NZ Data Futures Forum recently and suggested that NZ is like a start up in the global economy. We are small, agile and innovative. We have a very real opportunity to develop valuable IP not just in technology and techniques for handling this new era of data, but also in the way we manage privacy, identity, information and value. I'm excited to be part of NZ's play in this space and it's a key focus for what we're doing at Qrious.
