Scientists out of the Allen Institute for Artificial Intelligence (AI2) have recently developed a computer system that they report can solve standard geometry problems just like the average high school student (as per the SAT).
Advertisement
A score of 500, which indicates around 49 percent accuracy, may not sound like much, especially considering computers are supposed to be wildly good at subjects like math – after all, it’s just computation, right?
Researchers have combined natural language processing, computer vision and a geometric solver to come up with an artificial intelligence system dubbed GeoS that can solve SAT geometry questions like an average 11th-grade student.
These tests allow us to measure a machine’s ability to reason, and then compare its abilities with that of a human.
Etzioni said, “Much of what we understand from text and graphics is not explicitly stated, and requires far more knowledge than we appreciate”. He added that creating a system which will successfully be able to take the tests is quite challenging and they are very proud to have achieved the unprecedented results which they did. Though GeoS has only been tested on geometry questions, if the system’s accuracy was extrapolated, GeoS would have scored a 500.
“We’re excited about GeoS performance on real-world tasks”. However, some math questions have little hidden clues between the lines that might require the use of common sense and accurate language interpretation that might not go as deep for a computer.
In their paper detailing the results of their achievements, researchers explain, “Our method consists of two steps: interpreting a geometry question by deriving a logical expression that represents the meaning of the text and the diagram, and solving the geometry question by checking the satisfiablity of the derived logical expression”.
GeoS is the first end-to-end system that solves SAT plane geometry problems. From there, it processes something of an answer, then has to compare it to the available answers in order to produce a final response.
Complications arise in that SAT questions often contain unstated assumptions.
Advertisement
The researchers said that GeoS managed a score of 49% on its official SAT test as well as a score of 61 % on the practice questions. For example, diagrams may not be to scale or wording in text may be ambiguous (that may be understood easily by a human looking at the diagram).
