IBM's Watson Challenge No Gimmick

Register now

When IBM's Watson challenge goes live on television's Jeopardy game show, a 50-year slog of futurist computing credulity will again hang in the balance. The demonstration, billed alternately as man versus machine or as a natural language breakthrough, is also another stab at a durable vision of how people and computers ought to maturely interact.

Anyone who works with data and information will see Watson in part for what it is, a speech processing, question answering, talking machine built over four years at stunning cost to play the Jeopardy game show against human competitors. Reports so far have stopped at the implication of artificial intelligence from industry watchers burned too often by a term that has failed by definition to meet expectations in a single commercial product.

But even jaded technology analysts say what Watson does is a new thing that goes far beyond a chess match or computational exercise, and are tantalized by a new potential angle for the computing industry.

"Watson is a state of the industry example of what a vendor with a lot of imagination, R&D, experience and investment can do and I'm pretty sure it is going to be surprising and even shocking to people when they see it," says Charles King, principal analyst at Pund-IT Inc.

The upcoming TV event, already taped, pits Jeopardy uber-champs Ken Jennings and Brad Rutter against the computer in a three-match series. In a demo taped for reporters and analysts before the actual shows were recorded, the computer beat out the two champions.

On the surface, the event has the makings of a familiar gimmick, a talking video monitor atop a podium between two human foes. It evokes an episode of Star Trek or HAL from 2001, A Space Odyssey, not to mention Forbidden Planet and Robbie the Robot. IBM teased that imagery with a bit of anthropomorphic Jeopard-ese in the demo at a point when the computer tells the host, "Let's finish 'Chicks Dig Me,'" and draws a laugh from the audience before correctly closing the category.

For all that, long-time observers are struck by what behaves like a domain-specific but real personal digital assistant, authoritative, instantaneous, adaptive and standing by to take orders and answer questions with all the information at its disposal. What is surprising is how the system is so responsive, informed and hands off, with none of the awkward desocializing heads-down fixation consumers display with their personal devices.

"I'd almost call it a baroque approach to technology," King says, "a lot of different pieces pulled together on the computing side plus some enormous work on syntactical analysis and voice recognition and natural language."

Bits of all these technologies are common in smart phones and other types of computers but are much more esoteric in the depth Watson delivers. As a fine-tuned collection of parts, Watson is part hardware and processing, part software and natural language-processing, heavy on algorithms that parse vernacular and build associative relationships among the data stored within. IBM claims to have spent $3.2 billion over the last four years developing the products behind Watson.

Some have remarked that IBM didn't build a true computerized answer machine, but rather one that can play a game like Jeopardy at an expert level. But that is precisely the type of opportunity that has analysts excited.

Richard Doherty, who co-founded the consultancy Envisioneering with Steve Wozniak in early 90s, attended the live demo and came away convinced that the architecture of Watson is loaded with potential.

"It's a finite data set that does not include things outside the scope of the Jeopardy game, but it is such an amazing demonstration of near real-time competency that the mind just goes afire with the opportunity for other finite data sets," says Doherty. "You immediately think of construction or civil planning or health care planning where this kind of preloaded data can very quickly post answers." Third-party opportunities, he says, could be "phenomenal" for creating data sets that could open up "a whole new IT industry."

Analysts picture such a system in the hands of a physician seeking help with a diagnosis. The benefits might be staggering if a doctor were able to instantly inspect every bit of published medical research from the last three years and draw conclusions with an attached degree of confidence to compare to his own hunches.

More than speed, it is the confidence-building ability of Watson that has analysts seduced by the demonstration. Watson doesn't just hunt through materials like a search engine. It builds on its understanding of clues in the vernacular of Jeopardy the same way a puzzler would solve a Will Shortz New York Times crossword. It looks for associations, gathering evidence and sureness along the way. Unlike a search engine, the machine will not even offer an answer below a preset threshold of confidence.

Longtime data analyst Merv Adrian, who's now working for market researcher Gartner Inc., sees the Jeopardy wordplay as a classic learning challenge for man or machine. "Each column on a Jeopardy board gives an answer that suggests a type of question. You might get the first two answers wrong before you figure out the right way to ask the question."

And that's what Watson does, the analyst says. "It might miss the first question or the second, but with the right answers exposed, it uses dozens or thousands of algorithms to compute the scenarios for possible answers and having failed or assigned too low a weight, it recalibrates how it's evaluating the question. Every time it starts solving questions in a new domain, it learns more about which of the tools at its disposal should be used to evaluate."

That's the closest thing to real learning in a natural language environment that we've ever seen, says Adrian, and something that would be extraordinarily helpful in a diagnostic setting.

Doherty says the leap from 20 percent confidence to 70 or 80 percent confidence or more in such a use shatters a barrier. "It is the first time in all of my years of being involved in the IT industry I've seen a number like that." Were the machine not programmed to deliver subsecond responses for game play, he adds, the confidence level would immediately be higher.
The system has capacity for huge amounts of data in its in-memory database, meaning it is the equivalent of a PC with all RAM and no hard drive, extremely fast because it requires no external retrieval of information. For Jeopardy, Watson's "local" memory was loaded with the equivalent of 200 million pages of pop culture and reference documents and doesn't connect to external systems during the game. 

Bernie Spang, IBM's director of strategy and product marketing for database software systems, credits IBM's InfoSphere portfolio and other products already on the market as sharing core contributing parts of Watson. If not household knowledge, the products have been closely watched in IT circles all along.

Another publicized IBM campaign shows how the University of Ontario in Toronto hospital has been using InfoSphere Streams software to analyze 13 pieces of telemetry from neonatal care units to detect patterns indicating an infant might be in trouble and on its way to a life-threatening infection.

"They demonstrated the software is able to detect those patterns up to 24 hours faster than the doctors and nurses are able to," Spang says. "InfoSphere Streams was an IBM research project just three years ago and it's now a product we are selling in financial services, in government as well as health care."

For a system like Watson, IBM is also contemplating mundane use cases for businesses overwhelmed with product documentation and manuals for customer service, or legal or government services facing regulations. "When I have a legal question, I want to weigh all the laws and policies and regulations and precedents," Spang says. "This is just a natural extension of the idea."

With processing and memory prices continuing to fall, more and more projects will become feasible from a cost standpoint, and there will be opportunities for services, remote access and form factors much smaller than Watson's 10 "refrigerator-sized" computing racks. 

IBM may have gotten out in front of its own marketing with an ad campaign launch more than a full month before the event. Spang, on the other hand, feels the press timing is just about right.

"I believe in the next year or so there will be an IBM commercial with a doctor talking about asking their Watson system a question and getting an answer they wouldn't have gotten in weeks or months to immediately treat a patient and save a life, that will in no way surprise me."

Industry watchers are almost as aggressive with their timeline, and betting on success with their own high confidence and unusual consensus.

Adrian says IBM's customers are already coming to them with classes of problems that they wouldn't have thought about a couple of years ago "because they knew computers couldn't do this kind of thing then."

At least for this style of computing, Doherty says, "the opportunities for boosting people's confidence in the intelligence we thought computers would have over the last five decades is now finally on the cusp of what I think is going to be history breaking."
And King says, if anything, breakthroughs like Watson from IBM or from another tech lab will only get more interesting with long-awaited success finally in hand. "In a sense we are designing the student and then designing how the student is to be taught. It's a very holistic exercise, and you are creating the kind of student you want to answer the questions you're going to ask. But that doesn't mean the answers you're going to get might not be profound."

(Editor's note: Click on the following links to read our sidebar on Watson's technology specs or our recap of its performance on "Jeopardy!")

For reprint and licensing requests for this article, click here.