Data Management and Migration
The guide defines 20 big data/Hadoop terms.
Vendors are hard at work developing technologies that will help us make the leap to extreme data. Now Fujitsu announced it has achieved "unlimited scalability" with its Storage ETERNUS CD10000 architecture.
Splunk announced this week the general availability of Splunk Enterprise 6.2 and version 6.2 of Hunk: Splunk Analytics for Hadoop and NoSQL Data Stores. Hunk is also available through the Amazon Elastic MapReduce, or Amazon EMR, console, which is priced by the hour.
There's an interesting piece in Government Health IT on the National Institutes of Health's four big data initiatives currently in development.
Here are a few of the product announcements made at Strata to give you an overview of the direction products and partnerships are now trending, or at least leaning.
Underscoring yet again that business users are essential to driving both the data-driven business and the big bucks to big data vendors, Platfora's Ben Werther said in his keynote that big data's "center of gravity is shifting to the business analyst and that's a really healthy thing because the person who's analyzing the data should be much more in control of the data. But that's leading to multi-structured questions and new stack requirements are emerging."
The Department of Energy's Energy Sciences Network, or ESnet, is deploying four new high-speed transatlantic links to deliver a total capacity of 340 gigabits-per-second.
There is a fascinating piece in The New Yorker on Project Cybersyn as the foreshadower of big data use by governments. This was a Chilean government project and, as envisioned, remarkably foretold...
Yahoo!'s previous world record was 70 minutes using a large, open-source Hadoop cluster of 2100 machines for data processing. DataBricks, founded by the creators of Apache Spark, completed the Daytona GraySort, which is a distributed sort of 100 TB of on-disk data, in 23 minutes with 206 machines with 6,592 cores during this year's Sort Benchmark competition.
Big data is at the core of the movie and gaming animation business. Animators have to create, analyze, work with and store terabytes of data as a matter of routine. And that creates more than a few problems. But leave it to imaginative creators to create solutions, too.