Who Error-Corrects AI? By Charles Hugh Smith

In addition to the fact that AI systems are imperfect, they don’t care about the customers. From Charles Hugh Smith at oftwominds.com:

Part of why AI chatbots are so dreadful is we know the corporation / agency doesn’t care whether our problem gets resolved or not.

Please note: Of Two Minds subscription rates are going up this Friday 3/1/24 from $5/month or $50/year to $7/month or $70/year, so subscribe by Thursday if you want to lock in current rates. If you’re considering subscribing, please read the comment from Substack subscriber Kelly at the end of the post.

Click-bait-scary forecasts of hundreds of millions of jobs lost to AI are as ubiquitous as incompetent AI chatbots. Richard Bonugli and I recently took a more nuanced look at AI Job Challenges and Trends, with the goal not of throwing the baby out with the bathwater (i.e. concluding all AI is junk science) but of focusing on AI’s limits in real-world problem-solving.

We can summarize these limits in one question: who error-corrects AI? the intrinsic problem here is data harvesting machine learning–the essence of Large Language Model (LLM) AI and other machine learning approaches–is the illusion of precision: the model selects the correct diagnosis 95% of the time, but who’s going to error-correct the vital 5%?

Consider being a patient with cancer that receives an all-clear/no-cancer diagnosis from an AI processed scan. In other words, consider the consequences of the AI tool being wrong 5% of the time. In the case of cancer diagnoses, a wrong diagnosis can be a death sentence, or it can open a pathway to unnecessary treatments and surgeries.

The illusion of precision leads to fatal assumptions: if the AI error rate is “only” 5%, but the majority of the 5% errors are the most consequential, then the entire idea of basing accuracy on the percentage of correct / incorrect hits is grievously flawed. In effect, AI might be accurate on the 95% of cases with limited consequences and mostly inaccurate on the cases that really matter, but this reality is lost in the claim that it’s 95% accurate.

Continue reading