AI firms give New York no safety guarantee

Representatives of OpenAI, Anthropic, Google and Meta declined on Monday to guarantee that their AI agents will always follow safety guardrails, telling New York City Council members under oath that eliminating risk was not possible, Fox News reported.

The companies testified on 5 October before the Council's Committee of the Whole, which includes all 51 members and, in the Council's words, is "reserved for matters of significant citywide importance". The hearing began at 11 a.m. at City Hall, and roughly 40 members took part over the day, according to the Council. It was the first Committee of the Whole hearing since 2022, TNW reported. The Council had billed it as the companies' first public testimony under oath.

Who testified

Speaker Julie Menin's letters went to the chief executives, but the companies sent policy and safety staff: Morgan Dwyer, OpenAI's head of policy development and operations; Logan Graham, head of Anthropic's Frontier Red Team; Shane Cahill, Meta's director for privacy and AI legislation; and Alice Friend, Google's director of AI and emerging tech policy, the Council said. SpaceXAI, which the Council subpoenaed on 28 September, sent no one. "We are pursuing that subpoena in court," Menin said, amNewYork reported.

Three former lab researchers also testified. Jacob Coxon, who left Anthropic in September and previously worked at OpenAI, appeared at Menin's request, Bloomberg reported in a story republished by The Business Times. "On the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction," he told the Council. Daniel Kokotajlo, executive director of the AI Futures Project and a former OpenAI researcher, and Alex Turner, formerly of Google DeepMind, testified remotely under subpoena, TNW reported. Turner put the chance of an AI takeover at "roughly one in three".

No numbers, no guarantees

Menin asked each company to quantify the risk of a worst-case catastrophe. "I don't know. I also don't think it matters whether it's 1% or 10% or a 20% chance that something catastrophic will go wrong," Dwyer said. "None of these levels is remotely acceptable." Menin called the answer "flippant at best", amNewYork reported. None of the four gave a figure.

Asked to assure the public that their agents would always follow safeguards, Dwyer said: "It is not possible for me to commit or guarantee that any technology is without risk." Graham said the science of controlling the systems is "fundamentally hard and unsettled", and Friend said "to promise perfection would not be possible with any product on the market", according to Fox News. "I don't think we're asking for perfection," Menin replied. "We're just asking for accountability, transparency and safety overall."

The companies made no blanket commitment that failing an independent safety test would stop a release. They largely avoided saying whether they would bear legal liability if a system caused serious harm, though Friend said "if it's illegal without AI, it's still illegal with AI", amNewYork reported. When Menin asked who carried insurance against catastrophic risk, none of the four raised a hand, TNW reported.

Ten bills

Ten bills were on the agenda. Menin's lead bill would make it unlawful to market, sell or deploy an AI model in the city without third-party validation or a way for a human operator to shut it down, with a fixed $25,000 civil penalty per instance. Other bills would pay whistleblowers 25 per cent of recoveries, or 50 per cent if they bring the case themselves; let people sue AI providers for foreseeable harm from third-party misuse; require 24-hour reporting of AI safety incidents involving city contractors; and bar misleading safety claims in AI advertising, with penalties of up to $25,000, according to the Council. We set out the package in more detail here.

The committee laid over every bill, the Council's record shows. PIX11 reported that members expected to take them up again on Thursday, 8 October.

The hearing followed a voluntary White House accord signed by leading AI companies. "When the federal government fails to act, cities take the lead," Menin said in a statement. Dwyer told the Council that "voluntary commitments and self-regulation aren't enough", PPC Land reported. Coxon said measures such as kill switches "may be helpful in the short term", but that frontier development needed to slow down.