Program analyzes Web images in a quest for common sense

Common sense, which gets its name because it comes so naturally to humans, is not so common among robots. Researchers at Carnegie Mellon University, however, are trying to change that with a computer program called the Never Ending Image Learner, or NEIL.

NEIL is currently running 24 hours a day, searching the Web for images and trying to make sense of them. What makes the program special is that it goes beyond simply identifying images to actually trying to understand the natural relationships between objects — which is where common sense comes in. After analyzing enough images, NEIL begins to recognize that cars are found on roads while zebras are typically in savannahs. Researchers say the program has already learned 2,500 associations from thousands of instances.

“Images are the best way to learn visual properties,” Abhinav Gupta, assistant research professor in Carnegie Mellon’s Robotics Institute, said in a statement. “Images also include a lot of common sense information about the world. People learn this by themselves and, with NEIL, we hope that computers will do so as well.”

The university says a computer cluster has been running the program since late July and during that time it has analyzed “three million images, identifying 1,500 types of objects in half a million images and 1,200 types of scenes in hundreds of thousands of images.”

That alone has required some serious computing power — the program runs on two clusters of computers that include 200 processing cores — yet NEIL still has a long way to go. The team points out that Facebook alone holds more than 200 billion images, with more being added constantly, making the program’s task indeed never ending.

NEIL is possible thanks to advances in computer vision that allow for accurately labeling and cataloging objects with minimal human supervision. Essentially, the computer works away on its own, with humans only stepping in occasionally to correct mistakes. That level of efficiency is essential for compiling as large a collection of images as possible.

“What we have learned in the last 5-10 years of computer vision research is that the more data you have, the better computer vision becomes,” Gupta said.

The research is supported by Google — which obviously has an interest in the enormous visual database being generated by NEIL — and the Office of Naval Research.

The public can now view NEIL’s findings at the project website, neil-kb.com. In addition, the research team behind the project will present its findings on Dec. 4 at the IEEE International Conference on Computer Vision in Sydney, Australia.

[ image courtesy of Carnegie Mellon University ]