The future of big data: what's next probably isn't what most would guess


Big data has moved from a trend to a given, even though not all companies are comfortable with the tools yet. Indeed, many big data projects are still focused on the low-hanging fruit and the several fields of information in the harder data warehouse terrains are largely left unharvested. But things are progressing and everyone pretty much accepts that data-driven is the only way forward. The question is, what comes next?

Bernard Marr has a good post in Forbes on 17 predictions he thinks everyone should pay attention to now. Indeed, I would agree, even though I think some of those predictions are pretty much expectations already. Give it a read and see what you think.

As for me, I'm trying to look a little further down the road. I know most of you are too. I'll share my thoughts on what I see coming next and I hope you'll share yours too in the comments below.

We stop transferring so much data. We work on ways to get data out of siloes in order to be able to analyze it all. But I think we've managed only to create more siloes. We're transferring data to and fro, blending it here and there, streaming it, storing it, putting it in the cloud, moving it to hybrid clouds… well, you get the picture: data siloes again, just of a different nature. At some point I think we'll figure out how not to move so much data around so often and still be able to analyze it all as if it were in one place. How will we do that? That I don't know. Perhaps analytics or AI that go to the data rather than the other way around. Certainly analytics on the edge can transfer outputs to other analytics programs thereby circumventing some of the need for raw data transfer. We'll see what actually develops.

Outputs follow workers and dashboards go away – mostly. It's a wee bit nuts to have to open so many applications to retrieve data.  At some point, analyses will be constant, done in the background, and their outputs served automatically to the user at the right time. We see a bit of that emerging in the consumer space in connected cars that are already planned to serve data to the user just as they are going to a meeting or as a reminder to stop and get something they need while in route to somewhere else. But we've yet to see much of that development in the enterprise space. I think the next step in big data is outputs showing up on the appropriate device in context to what the employee is doing at the moment. When that happens, dashboards within applications will be of less use and appeal. Instead, machine learning becomes the ultimate office assistant and constantly follows its user around with the right information in hand.

Multi-factor data verification becomes a big deal.  We have two-step and multi-factor user authentication now to verify the user. Oddly, we don't have that to verify the data and analysis. As data breaches for data theft give way to data breaches for the purpose of data manipulation instead, we'll need a way to verify that the data and analysis is real and reliable. Hence, I think multi-factor data verification is coming up soon.

Rogue IoT data kill switches become a security necessity. As data breaches take on a more physical threat level, such as making dams flood a city, making a plane take a sudden dive, or making autonomous cars purposefully crash, legitimate users and operators will need an immediate means to override the automation's data feed. Yes, if you're in an autonomous car and it picks up speed while aiming for an embankment, it would be handy to have a way to immediately stop the thing and regain control. The same will be handy as a remote feature so that dams can be shut off immediately after a hostile takeover and so on.

Now it's your turn. What do you think is next on the big data agenda? -Pam@bakercom1