Businesses couple intuition with data for better outcomes

Tools

Yes, I know, human intuition is frowned upon in many data scientist circles and intuition can short-circuit big data efforts when it is used improperly. But to say it has no role at all is a mistake, too. Take for example, the story of True&Co, a startup using big data to engineer the world's best bra. It was humans who thought of that use for data--and to reinvent the business in the process. And it is humans that intuit other uses and meanings derived from data in other companies as well. Now along comes research from The Economist Intelligence Unit (EIU) that says those companies' instincts to combine both human intuition and data analytics were correct.

"They both play a role," Peter Swabey, senior editor, technology at EIU says in a CIO.com article. "The process of developing data is the process of trying to identify what the true state is. In identifying that, your intuition could be a useful guide."

Experts quoted in the article had interesting insights to offer on this dynamic, too.

"However much data you take in, and whatever the interview process, when you get to the end there has to be an element of gut feel too," says Alison Robb, group director for people, customer, communications and commercial at Nationwide Building Society, one of the expert practitioners interviewed for the EIU report in the previously mentioned CIO.com article. "It's partly chemistry, experience and knowing what you do and don't like."

Still, I was encouraged to hear from that report that companies that combine the two do not over-rate the importance of either.

"Perhaps nothing illustrates the inextricable link between data-driven decisions and human intuition better than how people would respond if the available data contradicted gut feeling," reads the CIO.com report. "Fully 57 percent of survey respondents said that if confronted with that situation, the first thing they would do is reanalyze the data; 30 percent said they would collect more data and only 10 percent said they would take the course of action suggested by the data."

Don't get me wrong--it's not that I'm preaching that data should be a backseat driver. I'm just saying that despite the size of data available today, much of it contains gaps in information and much of it is out of context. This means that humans must be able to recognize those short-comings and intuit faulty analytical outcomes in order to do the things necessary to correct them.

Blindly following either human intuition or big data analysis is a bad idea.

For more:
- see the CIO.com article
- see the EIU study

Related Articles:
No room for marketers' instincts in Era Of Big Data? Yes there is!
More voices in artificial intelligence debate are heard
Looking for big data and machine learning to replace human thinking? Ain't gonna happen
US military infuses human expert knowledge in algorithms