Asked who carries catastrophe insurance, all four AI labs kept their hands down

At the New York City Council's AI hearing at City Hall on Oct 5, Speaker Julie Menin asked sworn witnesses from Anthropic, OpenAI, Google and Meta two questions: what are the odds of a worst-case AI catastrophe, and who carries insurance against catastrophic risks? Each company sent one representative, appearing by video.

Asked for a number, OpenAI's Morgan Dwyer said the exact figure did not matter, because no level of catastrophic risk was acceptable. Menin called Dwyer's answer "flippant at best." Google's Alice Friend said forecasting catastrophic risk "is not a perfect science at this stage." Menin concluded that "neither of the four of you, no company here, can quantify the risk of something cataclysmic happening."

Two former AI lab researchers who testified the same day did give odds: Alex Turner, formerly of Google DeepMind, put AI takeover at "roughly one in three," and Jacob Coxon, formerly of OpenAI and Anthropic, said humanity losing control is "more likely than not."

Menin had opened the hearing by saying "The idea that artificial intelligence is going to self-regulate defies all reason." Menin's proposed bill would make it unlawful to market, sell or deploy an AI model in the city until third-party validators, meaning checkers outside the company that built it, have reviewed it, with a fixed $25,000 civil penalty per instance of an unvalidated model.

The second question was who carried insurance against catastrophic risks. The Next Web reports that none of the four raised a hand, and Fortune reports Menin's reply: "So then the public, I assume, will be asked to absorb the costs."

None of the four raised a hand, so Menin's assumption stands: the public absorbs the costs.

Sources: CNBC (Oct 5), The Next Web (Oct 5), Fortune (Oct 5)