“Sparkling Water brings best of breed open source machine learning to the Apache Spark Communities with MLlib and Apache SystemML”.
Advertisement
IBM created the Data Science Experience to extend the speed and agility of Spark to more than two million members of the R community through new contributions to SparkR, SparkSQL and Apache SparkML.
The Data Science Experience also builds on IBM’s current Data Scientist Workbench capabilities, which include connections to multiple data sources, and have more than 7,000 registered users.
Making sense of data can involve a wide variety of tools, and IBM is hoping to make data scientists’ lives easier by putting them all in one place.
Further details on the IBM Data Science Experience can be found on the company’s site. Real-time analysis of traffic patterns and product trends allows Bernhardt to now make rapid adjustments to product placement, pricing and availability status.
USA Cycling: USA Cycling Women’s Team Pursuit is using IBM Spark, Watson IoT, mobile and cloud to derive instantaneous insights leading to game-changing training strategies and racing tactics. With IBM Analytics on Apache Spark, SETI has been able to embark on a new Stellar Pair Eavesdropping campaign which enables the organization to look for potential communications between planets that might be orbiting in double star systems.
The new distribution enables all advanced analytics including batch processing, machine learning, procedural SQL, and graph computation.
IBM began investing in Spark a year ago and has put $300 million toward making it an “analytics operating system”, IBM said in a prepared release.
It allows them to train their models on data 1000 times larger and 100 times quicker than was possible with open source R and almost 2 times faster than Spark’s own MLLib.
Along with advancing Spark’s machine learning capabilities through collaboration with Databricks, the company behind the in-memory analytics framework, IBM also said it would open a Spark Technology Center while committing more than 3,500 developers and researchers to focus on Spark-related projects.
Fair enough, but given Microsoft’s acquisition of Revolution Analytics and its announcements yesterday around R Server for HDInsight and Hadoop now being powered by Spark, I have a feeling we may see parity there soon.
“Just as IBM played a critical role in the development of computer science, we can see many similarities today” said Picciano.
“Machine learning has become indispensable when it comes to big data analytics regardless of the solution used – Hadoop, Spark, R or anything else”, said Ritika Gunnar, Vice President, Offering Management, IBM Analytics.
IBM, Spark and R IBM, who, you may recall, made a splashy announcement, around a $300M investment in Spark support, at least year’s Spark Summit, today announced a major software deliverable from that initiative.
Advertisement
Picciano said that, following IBM’s announcement today, data scientists cannot only enjoy greater access to large data sets, but they will also gain the ability to work more efficiently with such large volumes of data.
