Development Since
Deep Blue's victory over
Garry Kasparov in chess in 1997, IBM had been on the hunt for a new challenge. In 2004, IBM Research manager Charles Lickel, over dinner with coworkers, noticed that the restaurant they were in had fallen silent. He soon discovered the cause of this evening's hiatus:
Ken Jennings, who was then in the middle of his successful 74-game run on
Jeopardy!. Nearly the entire restaurant had piled toward the televisions, mid-meal, to watch
Jeopardy!. Intrigued by the quiz show as a possible challenge for IBM, Lickel passed the idea on, and in 2005, IBM Research executive
Paul Horn supported Lickel, pushing for someone in his department to take up the challenge of playing
Jeopardy! with an IBM system. Though he initially had trouble finding any research staff willing to take on what looked to be a much more complex challenge than the wordless game of chess, eventually David Ferrucci took him up on the offer. In competitions managed by the United States government, Watson's predecessor, a system named Piquant, was usually able to respond correctly to only about 35% of clues and often required several minutes to respond. To compete successfully on
Jeopardy!, Watson would need to respond in no more than a few seconds, and at that time, the problems posed by the game show were deemed to be impossible to solve.
InformationWeek described Kelly as "the father of Watson" and credited him for encouraging the system to compete against humans on
Jeopardy!. By 2008, the developers had advanced Watson such that it could compete with
Jeopardy! champions. During the game, Watson had access to 200 million pages of structured and unstructured content consuming four
terabytes of
disk storage but was not connected to the Internet. as well as students from
New York Medical College. Among the team of IBM programmers who worked on Watson was 2001
Who Wants to Be a Millionaire? top prize winner Ed Toutant, who himself had appeared on
Jeopardy! in 1989 (winning one game).
Jeopardy! Preparation In 2008, IBM representatives communicated with
Jeopardy! executive producer
Harry Friedman about the possibility of having Watson compete against
Ken Jennings and
Brad Rutter, two of the most successful contestants on the show, and the program's producers agreed. Watson's differences with human players had generated conflicts between IBM and
Jeopardy! staff during the planning of the competition. IBM repeatedly expressed concerns that the show's writers would exploit Watson's cognitive deficiencies when writing the clues, thereby turning the game into a
Turing test. To alleviate that claim, a third party randomly picked the clues from previously written shows that were never broadcast.
Stephen Baker, a journalist who recorded Watson's development in his book
Final Jeopardy, reported that the conflict between IBM and
Jeopardy! became so serious in May 2010 that the competition was almost cancelled. To provide a physical presence in the televised games, Watson was represented by an "
avatar" of a globe, inspired by the IBM "smarter planet" symbol. Jennings described the computer's avatar as a "glowing blue ball crisscrossed by 'threads' of thought—42 threads, to be precise", and stated that the number of thought threads in the avatar was an
in-joke referencing the
significance of the
number 42 in
Douglas Adams' ''
Hitchhiker's Guide to the Galaxy''.
First match The first round was broadcast February 14, 2011, and the second round, on February 15, 2011. The right to choose the first category had been determined by a draw won by Rutter. Watson, represented by a computer monitor display and artificial voice, responded correctly to the second clue and then selected the fourth clue of the first category, a deliberate strategy to find the Daily Double as quickly as possible. Watson's guess at the Daily Double location was correct. At the end of the first round, Watson was tied with Rutter at $5,000; Jennings had $2,000. Watson also demonstrated complex wagering strategies on the Daily Doubles, with one bet at $6,435 and another at $1,246. Watson took a commanding lead in Double Jeopardy!, correctly responding to both Daily Doubles. Watson responded to the second Daily Double correctly with a 32% confidence score. However, during the Final Jeopardy! round, Watson was the only contestant to miss the clue in the category U.S. Cities ("Its
largest airport was named for a
World War II hero; its
second largest, for a
World War II battle"). Rutter and Jennings gave the correct response of Chicago, but Watson's response was "What is Toronto?????" with five question marks indicating a lack of confidence. Ferrucci offered reasons why Watson would appear to have guessed a Canadian city: categories only weakly suggest the type of response desired, the phrase "U.S. city" did not appear in the question, there are
cities named Toronto in the U.S., and
Toronto in Ontario has an
American League baseball team.
Chris Welty, who also worked on Watson, suggested that it may not have been able to correctly parse the second part of the clue, "its second largest, for a World War II battle" (which was not a standalone clause despite it following a
semicolon, and required context to understand that it was referring to a second-largest
airport).
Eric Nyberg, a professor at Carnegie Mellon University and a member of the development team, stated that the error occurred because Watson does not possess the comparative knowledge to discard that potential response as not viable. The game ended with Jennings with $4,800, Rutter with $10,400, and Watson with $35,734. In the first round, Jennings was finally able to choose a Daily Double clue, while Watson responded to one Daily Double clue incorrectly for the first time in the Double Jeopardy! Round. After the first round, Watson placed second for the first time in the competition after Rutter and Jennings were briefly successful in increasing their dollar values before Watson could respond. Nonetheless, the final result ended with a victory for Watson with a score of $77,147, besting Jennings who scored $24,000 and Rutter who scored $21,600.
Final outcome The prizes for the competition were $1 million for first place (Watson), $300,000 for second place (Jennings), and $200,000 for third place (Rutter). As promised, IBM donated 100% of Watson's winnings to charity, with 50% of those winnings going to
World Vision and 50% going to
World Community Grid. Similarly, Jennings and Rutter donated 50% of their winnings to their respective charities. In acknowledgement of IBM and Watson's achievements, Jennings made an additional remark in his Final Jeopardy! response: "I for one welcome our new computer overlords", paraphrasing
a joke from
The Simpsons. Jennings later wrote an article for
Slate, in which he stated: IBM has bragged to the media that Watson's question-answering skills are good for more than annoying Alex Trebek. The company sees a future in which fields like
medical diagnosis,
business analytics, and
tech support are automated by question-answering software like Watson. Just as factory jobs were eliminated in the 20th century by new assembly-line robots, Brad and I were the first
knowledge-industry workers put out of work by the new generation of 'thinking' machines. 'Quiz show contestant' may be the first job made redundant by Watson, but I'm sure it won't be the last. Drawing on his
Chinese room thought experiment, Searle claims that Watson, like other computational machines, is capable only of manipulating symbols, but has no ability to understand the meaning of those symbols; however, Searle's experiment has its
detractors.
Match against members of the United States Congress On February 28, 2011, Watson played an untelevised exhibition match of
Jeopardy! against members of the
United States House of Representatives. In the first round,
Rush D. Holt, Jr. (D-NJ, a former
Jeopardy! contestant), who was challenging the computer with
Bill Cassidy (R-LA, later Senator from Louisiana), led with Watson in second place. However, combining the scores between all matches, the final score was $40,300 for Watson and $30,000 for the congressional players combined. IBM's Christopher Padilla said of the match, "The technology behind Watson represents a major advancement in computing. In the data-intensive environment of government, this type of technology can help organizations make better decisions and improve how government helps its citizens." == Applications ==