As per experts, big-data analysis requires human perception for searching patterns and choosing which features of data need to be analyzed. For example, in a database containing the beginning and end dates of sales promotions and weekly profits the important information could be the spans between the dates as opposed to the dates themselves. To test the first prototype of their system, they enrolled it in three data science competitions, in which it competed against human teams to find predictive patterns in unfamiliar data sets. The Data Science Machine finished ahead of 615 out of 906 human teams.
Advertisement
On the third competition, the prototype’s prediction was 87 percent accurate. It took most data teams months to complete data analyses that were completed by the computer in between two and 12 hours.
A graduate student at MIT Max Kanter said he and his fellow researches see the Data Science Machine like a complement to our human intelligence. The human teams went through months of analysis to get the patterns right but the Data Science Machine’s predictions were just as accurate and the results were revealed in just two to 12 hours.
The new system was created by Max Kanter whose computer science master thesis is the foundation for the Data Science Machine, alongside Kalyan Veeramachaneni, thesis advisor and a research scientist at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL).
Many corporations, institutions, businesses and governments collect a great deal of data already-and often must avoid collecting particular data due to network, storage and sensor constraints-but breakthroughs in machine learning and meta-analysis such as the Data Science Machine would augment the already interesting job of being a big data scientist by adding another layer of automation. These are mostly defining the power-yielding aptitude of wind-farm sites or forecasting which students are in danger for a dropping out of online courses.
“The Data Science Machine is one of those unbelievable projects where applying cutting-edge research to solve practical problems opens an entirely new way of looking at the problem”, says Harvard University professor of computer science Margo Seltzer, who was not involved in the research, in a report by the Daily Mail. “The first thing you have to do is identify what variables to extract from the database or compose, and for that, you have to come up with a lot of ideas”, commented Veeramachaneni. These statistics weren’t directly collected by MIT’s online learning platform, but they could be inferred from data available. The study shows that the data Science Machine eliminates human intuitions for big-data analysis. Database is stored typically in different tables.
The machine itself is basically able to track correlations in relationships that are built using numerical indicators.
For instance, one table might list retail items and their costs; another might list items included in individual customers’ purchases. The machine will perform a number of operations such as importing costs from the first table into the second and matching different items with same purchase numbers and come forward with candidate features like total cost per order, average cost per order, minimum cost per order, and so on.
Advertisement
Kanter and Veeramachaneni will describe their Data Science Machine next week in Paris at the IEEE worldwide Conference on Data Science and Advanced Analytics. It then generates further feature candidates by dividing up existing features across categories.
