In murky weather, fighter pilots are prone to making critical errors.

They override the onboard instruments and rely on gut instinct instead. The results can be fatal.

Big data specialist Devin Deen uses the analogy to describe how executives need to incorporate data-driven insights into business.

It's all about trusting the instruments - or the data - instead of just relying on experience or intuition when making business decisions.

Advertisement

"Gut feel will always be there," he says. "That's intuition and that's why we pay people with wisdom and experience; however that needs to be tempered with facts and insights from the data and you're seeing more people accustomed to doing that versus just gut feel."

Deen, who counts NZ Bus, Air New Zealand, Sky TV, BNZ and Herald publisher NZME as clients of his data analytics business Enterprise IT, says New Zealand lags by up to five years in the use of data analytics in business.

He puts that down to a couple of things.

First, the technical skills needed aren't here in plentiful supply.

And in a small market where it's possible to know your users "well enough", it can be difficult to justify the extra investment, he says.

"Do nothing is often what's done because there isn't really a need to do better."

But with globalisation comes clients, and competitors, far beyond New Zealand's borders, he says.

His business focus is on helping companies that are open to newer technologies and have a culture of internal innovation and disruption.

"Companies that have been around for a while, if they don't disrupt themselves they will get disrupted, so I looked for companies that were in that position and were open to looking at how these newer technologies could be leveraged to get insights they couldn't from the traditional way."

Deen jumped into big data analytics after more than a decade working in businesses built on data warehousing and business intelligence.

Originally from Florida, he emigrated half-way around the world to New Zealand for love, leaving behind a career as an officer in the US Marine Corps.

He'd met his future wife in Singapore in the late 90s, where he was playing rugby and she was visiting friends.

There was a short stopover here before a job in Silicon Valley during the dot.com boom lured him back to the States for what became a four-year OE for the pair.

His final project before returning Down Under as a parent of twins was working on a multi-player war game for the military.

The "Netflix-isation" of customer experience is how Deen describes the traits of advanced analytics - a heightening customer responsiveness built around providing what customers want before they even realise they need it.

Living in San Jose in 1998, he was receiving DVDs in the post from Netflix when it was a start-up business.

Netflix has come a long way since then, acquiring a tremendous amount of data about its users while taking the content delivery online, he says.

If you want to be relevant you have to compete in that timespace.

SHARE THIS QUOTE:

"The mechanism of delivering you that media, that entertainment, has changed but the crux of that interaction has not changed.

"They killed [video rental chain] Blockbuster and they did it by knowing their customers, knowing what they wanted, regardless of if it was DVDs or now, more instant."

As customers, clever use of advanced analytics will result in increased responsiveness from the companies we interact with, says Deen.

"It's about understanding the customers better to enable better services and products.

"It's actually really, really motivated by customer engagement."

As the use of big data to drive advanced analytics becomes more commonplace, Deen warns that it can be a double-edged sword.

In the wrong hands it can potentially come up with the wrong results if the incorrect tools, models or algorithms are used.

"People are producing 'alternative facts' to use those words, but accidental alternative facts because they actually don't know how to drive it properly."

While the widespread use of advanced analytics is a reality in the US, Deen says New Zealand is still four to five years away from that tipping point. "It's the next chapter, that's all it really is.

"It's getting smarter, better, faster in making those insights and data driven decisions.

"To compete, you have to be data driven.

"The pace of how you need to interact with your customers has moved from weeks, to days, to hours, to minutes, to seconds.

"If you want to be relevant you have to compete in that timespace."