IQ test: 'Artificial intelligence system as smart as a four year-old'
Computing rug rat struggles with common sense
Email delivery: Hate phishing emails? You'll love DMARC
Researchers at the University of Illinois at Chicago have applied an IQ test to MIT's ConceptNet 4 artificial intelligence system, and determined it's about as smart as a somewhat-challenged four-year-old child.
The team used the Weschsler Preschool and Primary Scale of Intelligence Test on the system and found it performed reasonably well on vocabulary and recognizing similarities, but scored very poorly on comprehension.
"ConceptNet 4 did dramatically worse than average on comprehension - the 'why' questions," said Robert Sloan, professor and head of computer science at UIC, and lead author on the study. "If a child had scores that varied this much, it might be a symptom that something was wrong."
ConceptNet 4 has now been replaced with a smarter AI system, ConceptNet 5, but Sloan said its predecessor's performance highlighted one of the fundamental problems with generating true artificial intelligence. Such systems are have great difficulty generating what humans call common sense, since that all-too-rare capacity requires not only an extensive amount of factual knowledge, but also subjective facts we learn in life.
"All of us know a huge number of things," said Sloan. "As babies, we crawled around and yanked on things and learned that things fall. We yanked on other things and learned that dogs and cats don't appreciate having their tails pulled."
For example, he said it's easy to build a computer that knows the freezing temperature of water, but it also has to know that ice is cold, which is a much more subjective calculation. The team is now feeding the research back into AI development with the hope of creating more efficient software that can complement (or compete with) humans.
"We're still very far from programs with common sense – AI that can answer comprehension questions with the skill of a child of eight," concludes Sloan. ®
