Last night, I gleefully skipped celebration of Valentine’s Day, in favor of sitting rapt in front of the television to watch Jeopardy! mega-winner (and longtime friend of BCC’s Police Beat Roundtable) Ken Jennings go up against IBM’s latest massively parallel Artificial Intelligence engine, Watson.
The Atlantic has dubbed their coverage of the matchup “Liveblogging the robot takeover or humanity’s finest hour,” and it is hard not to read this confrontation in such sweeping, maybe-apocalyptic terms. Especially when there’s a Mormon in the mix!
But of course, as a geek, what fascinates me is the technology. When I first heard that IBM was undertaking this project, I was awed and incredulous. Having worked on artificial intelligence, specifically natural language processing and document search and organization, I immediately thought of all the unique challenges of the Jeopardy! format. It is one thing to pull up a list of possibly relevant web pages in response to a few keywords. But Jeopardy!‘s format involves so much more—a “who’s-who” list of the things that are most difficult for computers to handle: wordplay, puns, indirect allusions, jokes, and sometimes complex formats (for example, the “before & after” question type). A question featured on the mini-documentary that aired with the show last night illustrates this kind of challenge:
Answer: This trusted friend was the first non-dairy powdered creamer.
The response (it bears stating for a Mormon audience!) is “What is Coffee Mate?” Notice the classically Jeopardyesque wordplay: “trusted friend” is supposed to clue you in to “mate” in Coffee Mate. If you simply break up this clue into a jumble of keywords, as many internet search engines will do, it is hard or impossible to know what the exact response should be. Is it asking for the name of a friend who famously likes non-dairy, maybe some movie or novel character with that trait? Watson’s response in the documentary, which was depicting a trial run, was “What is milk?” This is a laughable response to any human, since “non-dairy” obviously excludes milk!
A similar gaffe happened last night, when Ken buzzed in and gave the incorrect response, “What are the 1920′s?” at which point Watson jumped in for the steal and said, “What are the 1920′s?” Oops! It was an error few if any humans would make. (Interestingly, if someone did, we would blame it on nervousness–something Watson doesn’t have to deal with!) But it makes perfect sense, since Watson, from what I understand, is not wired for speech recognition (it receives the clues by text message). So it would have no way of knowing what Ken had just said. Watson also flubbed an obvious Harry Potter question, which was just painful to watch. (Correct response: “Who is Voldemort?” Come on!)
Those glaring errors aside, last night started as a complete rout: curtains for humanity. (If humans can’t dominate on the topic of Beatles music, what are we good for?!) But Watson had a hard time with a category asking for the decade of various events, giving a window of opportunity for the humans to regain some dignity. It seems Watson didn’t understand what was being asked (the decade–not names or places or other facts). Last night’s episode only covered the first round (Single Jeopardy), with Double Jeopardy and Final Jeopardy each taking place on separate nights, tonight and tomorrow. (Much of the time was taken up with the behind-the-scenes vignettes, which were interesting, though I’m not sure audiences will appreciate more of them on the second and third nights.)
For more on the technology and story behind Watson, I recommend the NOVA episode dedicated to the topic, “Smartest Machine on Earth.” Use this thread as a liveblog/open thread tonight and tomorrow.
UPDATE (SPOILER): Watson wins! Now we just need Colbert’s handgun with a flag pin to run for president as a Republican, and totally dominate Romney in the primary, to make it 0-2 in the Mormon v. Machine wars.
 This is only partially true. Most search engines will group words that (probably) go together into n-grams that represent a single thought, or a specially associated phrase. So, for example, “shopping cart” is really one thing, even though (in English, at least) it happens to be represented by two words. It helps a search engine to group these terms together as “shopping cart,” rather than treating them as a “bag of words” with “shopping” and “cart” just thrown in the jumble separately.