> instead of having the work reviewed by a second person, you can have model + person
But the argument still applies then - if only one incident occurs where a catastrophic mistake was not spotted by the model failsafe, and if later investigations show that the mistake could've been easily spotted by a second human, a human will be installed as a failsafe for the failsafe.
I think the general problem is the following: if a human makes a grand mistake, it can usually be attributed to a temporal lack of care, or just random bad luck, or happened because the person was having a bad day, or... it is also generally understood that making a grand mistake will be such a shock for the person responsible that this person will most likely never make the same mistake again. On the other hand, if a machine makes a grand mistake, the intuition of the general public, trained by centuries of experience with techology, is that this machine will make the same mistake again and again and again, when prompted with the same input. If the model is not designed to learn from mistakes in production, this will of course actually be true.
But the parent's point is that "the model and the human probably make different errors".
That is frequently true. Take the job of a lifeguard for instance. A single mistake can be catastrophic, and yet we know that people have trouble staying completely focused for hours on end.
AIs have no trouble staying focused and they can be trained to spot drowing swimmers pretty well even on a crowded beach.
Having a human lifeguard plus an AI that alerts the lifeguard when it spots something suspicious could lead to better outcomes than employing two lifeguards.
> Having a human lifeguard plus an AI that alerts the lifeguard when it spots something suspicious could lead to better outcomes than employing two lifeguards.
I disagree. I think you'll see the same thing as with drivers falling asleep while Tesla autopilot is running. The lifeguard will let the computer do all the monitoring since a low false negative rate combined with a low incidence rate means that most bodyguards will experience the computer being 100% reliable for weeks or months at a time. In fact it's not unreasonable that if the lifeguard sees someone drowning but the computer doesn't register it as such the lifeguard may question their own judgement based on experience.
This is very dependent on the false positive rate. Similar to the examples above about false negatives, if the AI gives false positives the lifeguards will stop paying attention to its feedback.
Huh? In my experience, people who make a mistake tend to make the same mistake again and again. Whereas, when a machine makes that mistake, somebody fixes the bug and it doesn't happen again. If the machine can teach itself, then so much the better, but "has someone supporting it" is a sufficient form of "learning from itself in production" to address the concern you raise. In the industry where I've been working for the last past several years (finance), when something goes wrong, the first instinct everyone has is to bump up the prioritization of automating whatever human task was responsible for the human error. And the automation is pretty much only bounded by the amount that the company is willing to spend on programmers and its ability to find good programmers who are willing to work for it and its insistence on maintaining backwards compatibility with bad legacy systems.
> Whereas, when a machine makes that mistake, somebody fixes the bug and it doesn't happen again.
Well, that is kind of the problem with AI. How do you fix a "bug" (if you can even call it that) in a model you cannot fully understand? Do you re-train it on the catastrophic mistake and somehow give it more weight? How can you be sure that this won't lead to any problems where previously there were none? How do you explain to a customer that your model now doesn't make mistake A anymore, but now mistakes B and C frequently occur? The only safe bet is to write some auxiliary code, which first uses the AI as a black box, and afterwards explicitly checks the result for this particular mistake. If this happens again, and again, and again, you need a human to maintain and extend this auxiliary code and also adjust it to changes in the underlying model, at which point I am quite certain just using a person of average intelligence instead of AI will be cheaper, more reliable and more flexible.
> But the argument still applies then [...] a human will be installed as a failsafe for the failsafe.
Surely this will depend on the cost of failure, and the cost of the human failsafe.
Spellcheck in an email client helps prevent the minor embarrassment of typos and spelling errors in emails, and few emails are so consequential that it's worth having them carefully manually vetted.
Of course, this is why I wrote "catastrophic" above. I also would not call an email spellchecker a business use case. Regarding spellcheckers per se: I have worked with book publishing companies in the last few years, and I can assure you all major book publishers employ real humans for the final spellchecking before a book goes into print.
But the argument still applies then - if only one incident occurs where a catastrophic mistake was not spotted by the model failsafe, and if later investigations show that the mistake could've been easily spotted by a second human, a human will be installed as a failsafe for the failsafe.
I think the general problem is the following: if a human makes a grand mistake, it can usually be attributed to a temporal lack of care, or just random bad luck, or happened because the person was having a bad day, or... it is also generally understood that making a grand mistake will be such a shock for the person responsible that this person will most likely never make the same mistake again. On the other hand, if a machine makes a grand mistake, the intuition of the general public, trained by centuries of experience with techology, is that this machine will make the same mistake again and again and again, when prompted with the same input. If the model is not designed to learn from mistakes in production, this will of course actually be true.