The UK Post Office deployed Fujitsu's Horizon point-of-sale system in the late 1990s. It had bugs that created false cash shortfalls. The Post Office prosecuted nearly a thousand postmasters for theft over 15 years. Fujitsu and Post Office staff testified in court that the system worked correctly. At least four people died by suicide. Nobody is calling this a database ethics failure, because it wasn't one. It was institutional fraud, enabled by software, carried out by people.
Benedict Evans uses this as the entry point into a precise argument: regulating 'AI' as a category is the wrong level of abstraction. The problems are not unified. Parole algorithms, mortgage approvals, drug discovery pipelines, and middle-school essay graders are all called AI. They have different failure modes, different stakes, different required expertise, and different legal frameworks that already exist to govern them. Google Gemini's image generation controversy has nothing in common with training data protocols for multiple sclerosis research. Evans invokes Larry Tesler directly: AI is whatever doesn't work yet. Once it works, it's just software, and everything is software now.
The car analogy is where the argument sharpens. Automotive regulation covers dealer contracts, crash safety, urban congestion, drunk driving, and fuel security, but not under one law or one agency. Evans argues tech regulation defaults to exactly that omnibus mistake. The harder problem is timing: it took 75 years to mandate seatbelts, and nobody wants to wait that long, but we are effectively writing aviation law in 1910. The full piece is worth reading because Evans does not conclude with a policy proposal. He concludes with a demand for humility, and explains precisely why that demand is technically justified, not just politically convenient.
[READ ORIGINAL →]