Speaking in tongues about big data

Tools

I am like the IRS. I automatically respond to certain red flags. Superlatives are one example. Anytime something is billed as unique, the red flag flies. Same with terms such as "real time", "plug-and-play" and "disruptive." There is another category that goes beyond the superlatives born of well-meaning efforts at marketing a product, and that is hyperbole. Hyperbole takes marketing spin to a whole new level, one that I am uncomfortable with.

Big data gets its fair share of hyperbole. That makes me uncomfortable because I have seen it before with other technologies and don't believe it's helpful. It's fun, but it doesn't always end well. I used to enjoy Jeff Pulver, one of IP communications (VoIP) greatest showmen as he tweaked the noses of big telecom carriers with his talk of disruption. I still do. But in the end, much of his message turned out to be hyperbolic and in the end deflated like a sad balloon.

Last week in O'Reilly Media's preview for its Strata Conference--which takes place next week in Santa Clara--Alistair Croll, entrepreneur, author, and host of the Strata events, admitted to his use of hyperbole in speaking about big data. Well, he admitted that some people believe it to be hyperbole when he talks of things such as the business singularity. I am one. I believe it.

However, I don't think in this case that it's all bad. At least it is well-informed hyperbole. And, it technically won't remain hyperbole forever--at least according to people much smarter than me who know what the singularity really is, what it will require to reach it and how we will know when we have or if we've crossed it or been blown away by it. I'm still unclear on all that.

Some of what Croll says sounds like science fiction, but there is a lot of machine learning going on with big data that could escalate exponentially and make you wonder if he isn't just being prescient.  I don't think we are anywhere near that level of artificial or machine intelligence, but it sure makes big data fun to think about. And that we are even scratching at that door makes this an exciting space to be in.

When he explains how the reliability and performance of software and its decision-making prowess can outperform in, say, a Human Resources department, it makes you wonder how close we are to relinquishing some control.

"Managing humans might be messy; it's fraught with emotion and governed by employment law, but nobody cares about pitting algorithms against each other. There is nobody is out there unionizing algorithms, so they fight in a battle to the death," Croll said. "Humans retire, but code just gets a faster processor. Yes, that's pretty controversial and troubling, but it is the way many people view software. Analysis and optimization lead to a continuous loop of improvement that gives us this exponential function for business."

Who can say no to that?

Croll says some companies like Google (NASDAQ: GOOG) are already on the other side of the singularity because they have changed how they view their priorities. They don't worry about things like scale anymore. They worry about cycle times, in particular about cycle times for learning.

"To them everything is an experiment, a chance to optimize. They analyze everything and they feed this back into themselves, continuously engineering their improved successor," Croll said. "These businesses on [the other] side of the singularity will seem opaque to us, like they are shifting and transient and unthinkably agile, and to them, traditional businesses will seem sluggish and predictable and unwise."

That's a little spooky, but it gets a bit easier to see where the hyperbole starts and ends, and that this may not be a bad way to view how one's business needs to change.  

Croll admits that "maybe it isn't as tectonic a shift as the rise of sentient machines or the middle of a black hole, but it certainly is more than just the evolution of business. It is the migration from a physical world to a digital one. We are moving from a business ecosystem where those with scale win, to one where those who have better cycles of adaptation and learning win."

Okay, I'll buy that. - Tim

Filed Under