Pat Pilcher: Big data, big opportunities

Back in March, Telecom Digital Ventures set their sights on the big data market and launched a new business called Qrious whose aim in life is to help organisations to wrestle some value from the sea of data and analytics they're currently drowning in.

I caught up with Cyrus Facciano, the GM of Qrious to see just what the deal is with this whole 'big data' thing. Turns out that it's bigger than big, its potentially huge as are the opportunities for New Zealand.

PP: So what exactly is big data?

CF: Big data is largely a phenomenon borne from technology. Technology that is generating vast quantities of data through sensors, mobile phones, and increasing engagement on the internet. Also, new technologies that enable us to do more with that data at a lower cost, faster and in a more flexible way.

PP: So it's pretty important to businesses right?

CF: It's important to businesses as it not only provides new opportunities to gain new insights into their customers behaviours and into their business performance, but also lowers the typical barriers to entry that have been historically associated with gaining insights from data such as cost and capability.

PP: How has big data made a difference to businesses?

CF: The concept of big data is still relatively new and uncharted, so it's still emerging but there are a few. For example;

* The now blueprint for recommendation engines at Amazon (customers who bought this also bought this)

* The famous Target example where they were able to detect that a teenage girl was pregnant due to her spending habits, to GE investing a billion dollars for a 1% gain in efficiency across their business

* A large travel company changing it's definition of loyalty from "repeat business" to "digital evangelists" based on their social media and web comments

* Healthcare organisations offering better care, more efficiently, based on better insights about their patient.

PP: Do many businesses get the whole big data thing yet?

CF: Not many no, like anything emerging, it's ripe for misunderstanding and inflated expectation. The challenge with this concept with data is that it's highly receptive to the "art of the possible" in that it doesn't take much for the imagination to run wild with the possibilities data can deliver. A lot of these possibilities are unrealistic or complex to achieve. That's why we think the first step is one of education, going back to the rule book and helping customers understand more about the data they have before getting too carried away with what's possible.

PP: It must have pretty big impacts on corporate IT requirements?

CF: Absolutely, that's why it has such a close marriage with Cloud Computing where companies can "rent" IT resources on a "pay-per-use" model. Requirements in the data space require a lot of power at irregular intervals of time, and ultimately are experimental in nature, so being able to align cost with this variable demand and to be able to fail with very little regretful spend is key to success without breaking the bank.

PP: You've previously spoken of data scientists as an emerging high value skill, how will that work?

CF: I'm a firm believer in a healthy balance of traditional analytics use cases such as data warehouse and business intelligence and the experimental side of data. Data Scientists operate in that experimental side, testing hypotheses, looking for correlations that answer questions the business is asking, usually using data from outside the organisation to find an edge.

PP: Do you have any examples of how big data can make life easy for customers?

CF: This is where the art of the possible starts to come in! There are literally thousands of examples, from improving transportation by amalgamating data from multiple sources to suggest the best route to take, to gene sequencing that looks for patterns of mutation in diseases for more effective treatment, to the more commercial use cases like helping you choose the best product to buy, or reminding you to grab your mobile phone charger because when you leave the house you would normally take it with you.

PP: Privacy and security must be huge issues though?

CF: Yes, it does tend to dominate most discussions on the topic, but it's also a huge opportunity as this area is still largely nascent in this new data context. That's why groups like the NZ Data Futures Forum are emerging because it's something that needs to be discussed. In my view, I don't think privacy legislation is necessarily the right path. I think a public opinion approach is more appropriate as it's constantly changing.

Ultimately, customers want to have the choice to share or restrict their personal information based on what value they're obtaining. It needs to be a continuously open dialogue. That's why I really like the concepts around dynamic privacy where customers can choose or set their privacy based on their preferences and change it if they want to.

In terms of security, it's a global issue seldom discussed. Bottom line is, there are more hackers in this world than security professionals to combat them. We need to work smarter. That's why I think we need to consolidate our information and protect it with the few security experts we have. The more dispersed it is, the more attack vectors hackers have to get information.

PP: Will the Internet where everything is connected impact on big data?

CF: Absolutely. Machines don't sleep! Machine generated data already exceeds human generated data, and as we create more machines that generate data that will increase even more. This will introduce both more opportunity as well as complexity in this space.

PP: Do you see any the future opportunities for New Zealand with big data?

CF: Definitely. I spoke at a recent NZ Data Futures Forum recently and suggested that NZ is like a start up in the global economy. We are small, agile and innovative. We have a very real opportunity to develop valuable IP not just in technology and techniques for handling this new era of data, but also in the way we manage privacy, identity, information and value. I'm excited to be part of NZ's play in this space and it's a key focus for what we're doing at Qrious.