Recently we had what was called a Nationwide Ground Stop, in which the FAA―citing software and hardware issues―stopped every plane in the country from taking off. It was a Friday the 13th, so perhaps they can blame plum bad luck or Jason Voorhees.
But Malcom Kyeyune has found another factor: the pursuit of efficiency over resilience. Which is to say, the demand that things get done now with fewer resources, while even the very near future has to fend for itself.
In the realm of systems theory, a system can either be resilient or brittle when dealing with various problems and negative conditions. Resilient systems degrade slowly and gracefully, and tend to have built-in failsafes or counter-balances. Perhaps the best example of this is the human body itself, which possesses the nearly limitless number of tricks, adaptations, and self-repair routines necessary to deal with everything from blood loss, infections, broken bones, temperature swings, starvation and so on. A brittle system, by contrast, tends to shatter like a pane of glass: due to how it is designed, it is subject to rapid, catastrophic points of failure once its tolerances are exceeded.
As a technology, AI is an inherently brittle system, and it cannot be anything but. The primary reason for this comes down to what AI is supposed to do, which is to save money and increase efficiency. In vernacular language, “efficiency” is always a good thing, and more of it is supposedly always better. But in systems theory, efficiency is inherently oppositional to resilience. A fire station that buys three fire trucks to keep in reserve in case the one it is actually using happens to break down is clearly spending money inefficiently, but it is also one that is far safer to be around compared to one without spare trucks. Similarly, a company that fires nine out of 10 human staff because the AI can handle their tasks might be more efficient, but it has also gone from being a resilient company to a brittle one. Even during flu season, the chances of all ten employees calling in sick is close to zero: the chance of the AI computer system experiencing a failure that completely stops it from working, on the other hand, is far higher.
This corresponds with what I've come to understand through my own observations, which is that jobs aren't being automated so much as they're just being eliminated. Divisions that once had ten employees, all checking each other's work, now might now have one guy and an "autonomous agent", while everyone just hopes for the best. If you go to a website of a company that's replaced its customer service department with AI, you'll find that said AI doesn't really do anything except restate the set answers on the FAQ list.
When you don't have enough coders because you've chosen to stop hiring them, the AIs which train on human-generated data start cannibalizing themselves and you get model collapse. If you don't have enough air traffic controllers and are trying to patch things over with bots, you could get something more serious. This isn't necessarily inevitable, but no one seems particularly invested in preventing it.