OpenAI recently cancelled the launch of a new frontier model after internal testing flagged safety failures, as reported on 28 September. The withdrawal illustrates how even leading developers can encounter unresolved hazards before public release, prompting renewed debate over the adequacy of existing safeguards for advanced artificial-intelligence systems.
The letter in The Guardian calls for independent oversight and regulation, yet it notes that no concrete framework has been offered to define how regulators would assess safety or what evidence would satisfy them. Without clear criteria, the suggestion remains a generic appeal rather than a practicable solution.
Engineering disciplines that manage safety-critical software, such as aviation control systems and nuclear power plant operations, rely on internationally recognised standards that demand comprehensive safety cases. These cases require systematic identification of hazards, demonstration of controls, and quantitative proof that the chance of a catastrophic event is vanishingly small.
A typical safety case must demonstrate, with at least ninety-nine percent confidence, that an accident capable of causing multiple deaths will not happen more often than once in a thousand years. This statistical target sets a very high bar for proof, demanding evidence that the probability of such an event is exceedingly low.
Providing proof for such low probabilities proves difficult even for aircraft, which at worst could kill up to one thousand people in a single crash. The aviation industry still struggles to meet the one-in-a-thousand-years benchmark, highlighting the practical challenges of quantifying ultra-rare catastrophic risks.
Developers of frontier AI systems claim their models could pose existential threats to humanity, yet they have not produced detailed risk analyses or safety cases that independent reviewers could evaluate. Moreover, no evidence has been offered that such safety cases are feasible for AI of this scale, leaving a gap between asserted danger and demonstrable mitigation.