This was answered in the Nova special, in passing. The question is fed to Watson as text at the same moment it appears on the display that the contestants can see. Likewise for correct answers. Watson is not performing speech recognition.
Aw this seems rather half-assed to me. I hope at a later time they do another one of these with a new and improved Watson. I really want to see a humanoid robot standing on the podium (connected by wifi to the supercomputer) and using OCR and speech recognition. I also think Watson's voice should be more authoritative. It's like they went to extraordinary effort to make Watson sound unimposing.
It can always be "awesomer". However, calling it half-assed I think is a bit harsh. The truly novel aspects of Watson are that in its abilies in to answer questions in a truly domian-free way. It is not trained to be an expert in a specific domain, but instead to calculate what it believes to be the best answer from any domain. This is truly novel and amazing. That it can do this with Jeopardy questions, regardless of how it is given those questions, is truly remarkable. Jeopardy questions are often times laden with puns or other literay devices. Watson can understand and answer better than the vast majority of humans on this planet. That it can't currently "see" or "hear" is, IMO, less interesting given the advances it makes in deep QA.
I didn't mean to sound harsh - I understand the significance. I just meant the technology involved in Watson being able to answer such questions is undoubtedly orders of magnitude more complex than the OCR and speech recognition required to process the question in the first place.
They're playing a "human" game after all, and they're not playing like a human is fed into the computer electronically.
You realize that if it were only to recognize speech, other players would have a significant time advantage because they can read the text in probably 1/10th of the time it takes the host to speak the question out loud. "Feeding" Watson the text is just another way of saying Watson "reads" the text displayed on the screen, just like every human is allowed to.
Yeah, OCR (optical character recognition) would probably be used for reading the questions. The speech recognition would be for listening to other people's answers so as not to duplicate wrong answers.
Well is the text "fed" in a word at a time, or the whole question as a block that takes nanoseconds to parse. That's an unfair advantage if so because because the computer is getting the data in a different format than the other contestants, who are handicapped since they have to preprocess the data in a way the computer doesn't because it has been preprocessed in advance for it.
How are the two situations any different? Just because a program has an entire block of text in memory doesn't mean that it can instantaneously build all the data structures needed to process and make sense of it. We both "read," Watson's "reading" just takes place in its code.
It's very different. The humans have to decode the visual and aural representations before parsing for meaning can commence. The computer should have to do this too, but that step has been done in advance for the computer. This gives the computer a time advantage not because it was faster at computing, but because necessary computing was removed in advance from its task.
I kind of agree, but OCR with a stable camera on the clue board would be extremely fast, reliable, and trivial, so it wouldn't make it much more impressive. Voice recognition (without OCR or a text interface to fall back on) would certainly be impressive, but I doubt it would be very reliable or impressive. I may be underestimating the state of the art on voice recognition; it may be very accurate considering Trebek's great diction and the clean studio audio that would be captured.
That's ironic, because "HAL" is a shift of IBM:
[I B M] - [1 1 1] = [H A L].
Apparently Arthur C. Clarke and Stanley Kubrick didn't notice this. "As it happened, IBM had given us a good deal of help, so we were quite embarrassed by this, and would have changed the name had we spotted the coincidence." - A.C.C.
The documentary mentioned that Watson does some speech recognition. It attempts to listen to the competitor's answers so that it doesn't give a wrong answer that was already given.
No, it is fed the correct answer by text once the question is completed. They said "listen" in the documentary, but later, in passing, specified what they meant by that. In particular, the demonstrated the active learning when Watson failed to understand a category required dates as answers. After 4 it learned that dates were the type of answer required and got the last question correct. Again, the feeding of the correct answers was happening after the question completed and via text, not speech.
You're talking about two different things -- SMrF is saying that if Jennings or Rutter gives an incorrect answer, Watson is able to run speech recognition on what they said, ruling out the answer they gave. This would happen before the question is completed.
So what translates the speech to text? I guess I just assumed they used software but perhaps there is a human doing this. I probably just missed when they mentioned this.
The correct answers are already known. Watson presumably gets the same answer the host has in front of him.
Someone would have to transcribe what the other contestants are saying only if Watson would also get their incorrect answers which doesn’t seem to be the case.
The contestant said "20s", IBM Watson said "1920's" - both were wrong but one can see how Watson didn't recognise that the answer was the same, it wasn't.
What about video clues? I assume there is a transcript, but sometimes there's a visual reference. Perhaps some sort of integration to Tineye.com or advanced video recognition? :-)
I would think as soon as it computes an answer. Of course there is a penalty for getting an answer wrong, so it can't be pressing the button all the time. But the PBS documentary I think hinted that it will be a sent a signal to know what amount to wait before reading aloud an answer.
That said, I wish the engineers would have implemented OCR technology into Watson so it could see the TV screen, decipher the text question, and understand it -- instead of directly being fed the question. It would have given Watson a nice little title of being'complete'.
What he means is that you aren't permitted to buzz in on Jeopardy until the host has completed reading the question and the buzzers are activated. Perhaps Watson is spamming the 'answer' button from the moment he knows the answer, or perhaps they let him cheat by sending him a signal when the buzzers become activated. Given that the process of buzzing in to answer a question is a large part of the strategy of the show though, I hope they aren't letting him cheat in that way.
Edit: Failed to read the comments lower on the page. Looks like Watson plays by the same rules as everyone else, using a physical buzzer which can't be activated until the question is read and the 'answer now' signal light is activated. I know that another restriction on when to answer that Watson has is that he won't buzz in until he's confident in an answer, while human players will sometimes buzz in figuring they will be able to come up with the answer in the couple seconds between buzzing and answering.
I'd been told by a competitor on the show that timing the button press correctly was an underappreciated part of the game. If you press the button before you're allowed to (before Mr. Trebek finishes reading the question) your button is locked out for long enough (maybe a second?) that you're unlikely to get a second chance at that question. So I assume Watson wouldn't be allowed to "spam" the button.
In the case of Deep Blue the movement of the pieces isn't pertinent to the game - if human/robot assistants moved all pieces for both players it wouldn't alter the result [except possibly in some very edge cases].
For IBM Watson playing Jeopardy against humans the buzzer reaction time is exceedingly important. The buzzer is an integral, possibly even central, part of the game.
