IBM answer machine makes chumps of trivia chimps
'Robots will keep us as pets'
If you've been watching the first two days of the Jeopardy! game show pitting two of humanity's trivia champs against IBM's Watson question-and-answer machine, you probably had a sinking feeling – mixed with a sense of awe.
All-time champ Ken Jennings put up a good fight, and Brad Rutter stole away a few questions, but the supercomputer took no prisoners in rounds three and four of the tournament. The pace was twice as fast on Wednesday, with less marketing speak on behalf of Big Blue, and they got quickly down to quizness.
Here's how the third round played out:
After a slow start, Jennings worked up some momentum and Rutter did his best to keep up. By the end of the round, Jennings had amassed $8,600, Watson only $4,800, and Rutter $2,400.
In round four, it was a slug-fest between Jennings and Watson. But as you can see below, it wasn't enough:
By the end of the fourth round, Jennings had $18,200, Watson had $23,440, and Rutter had $5,600.
The show closed out with the Final Jeopardy! clue: "An Account of the Principalities of Wallachia and Moldovia inspired this author's most famous novel."
Rutter, who was in third place, bet $5,600, everything he had, and wrote "Who is Bram Stoker?" That boosted him to $11,200 for the day and $21,600 for the three-day tourney. Jennings bet a mere $1,000, and his response was "Who is Stoker (I personally welcome our new computer overlords)?" That gave him $19,200 for the day and $24,000 for the week. Watson, ever the perplexing wagerer, bet $17,973 and responded, "Who is Bram Stoker?" That gave him $41,413 for the day and $77,147 for the entire Jeopardy! challenge.
As Oren Etzioni, director of the Turing Center at the University of Washington, put it in a report on National Public Radio on Monday ahead of the tournament: "Does that mean that it's Game Over for humans, that robots will keep us as pets?" Etzioni says no. We say: "Middle managers, get used to the dog food." ®
Obvious joke alert
There already are humans kept as pets by machines - they're called "iPhone owners"!
I'm here all week, unfortunately.
Congrats to you, Captain...
...for demonstrating in one brief post how little you know about Watson, computer chess, AI, programming, and human intelligence. Quite a feat!
Watson is dramatically different from chess-playing computers. It has to figure out the meaning of an ambiguous and convoluted natural language sentence, then come up with a context-relevant answer. This is *very* hard, and Watson does it better than the two best humans ever to play the game.
By contrast, computers play chess largely by brute force. Sure, there is a significant amount of algorithms guiding and pruning the tree search, but the way computers play chess bears no relation to the way humans do, and consequently sheds no light on human intelligence.
As for your comment that "the ability to reason for any machine is only as good as the programmer setting about to create the rules for how to reason", that is just flat out wrong. It is a complete non sequitur. Programmers have been writing expert systems for decades that can reason about domains that their programmers know nothing about, and do it better than human experts. You're confusing the rules with the outcome of the rules. It would make as much sense to say "the ability of a robot to weld joints on a car is only as good as the programmer setting out to create rules for how to weld" (and by the way, most programmers are crap welders). This is obviously and demonstrably absurd, but somehow, when the skill under threat is reasoning instead of welding, otherwise-intelligent people go all moony.
As for your reference to "REAL artificial intelligence": what would you define as "REAL"? I'm guessing that it's along the lines of "whatever computers can't do yet"...
An advantage or otherwise was never really the point
Exactly. And even if Watson did have an advantage when it comes to button pressing speed, it would be cancelled out by the fact that it doesn't receive the text input until Trebek finishes reading the question.
All that time while Trebek is reciting, the human players are reading the whole question from the screen. They're already formulating possible answers, determining how confident they are, weighing up the risk of buzzing in against holding back. Watson doesn't get to begin to do any of those things until Trebek finishes speaking and the text is fed in. Watson is fast but the time taken for its computations is not zero. In many cases one of the human's switches will be well on its way to hitting the contacts before Watson has even begun to make sense of the question.
In fact there was strong evidence of this during a couple of the last day's rounds, especially the one about computer keys which, ironically, Watson flubbed badly. In many cases Watson's potential answers appeared on screen a distinct fraction of a second after one of the other contestants buzzed, and quite a few of the answers it had come up with were wildly off base. It actually played so badly in a couple of those early rounds I thought one of the opponents had sneaked into the server room and yanked a few boards.
But we're missing the big picture here. Jeopardy is just a canvas on which this experiment has been played out. You could tweak the rules to benefit the machine or the humans, or you could tweak Watson's algorithms to make it better and better at Jeopardy and witness the law of diminishing returns with each iteration.
But it's not about building a computer that can play TV quizzes, fun though it's been to watch. It's about building a machine that can process 'human-formatted' information in a way that machines have never been able to do before.
And that's an amazing achievement. One which opens up almost limitless possibilities once implemented in other fields.
Imagine a Watson-like system freed of the limits imposed by the game show format. One that could be fed dozens of related queries at once, not just a four- or five-line quiz question. One that didn't have an artificial time limit placed on its ability to calculate the best answer, but could cogitate on it for hours or days. One that wasn't limited to its built-in database but could use the best answers it had already come up with to go out onto the internet and search for more information, refine its answers further and use those to search deeper and so on. The possibilities are staggering.
I cannot understand how anyone, especially anyone reading a technology website, can not be blown away at just what a stunning achievement Watson is. Ever since a teacher brought a ZX80 into my school three decades ago and I marvelled at this little programmable box, I have been following advances in the world of computing and been ever more amazed at every turn. And Watson is by far the single most impressive thing I have yet seen, by a massive margin.
Kudos, IBM. With Watson you have entertained me and you have awed me and I think you may genuinely have shown me a glimpse of the future.