New York's council wants a validator for every model

The most consequential document from New York City Hall on Monday was not the testimony. It was a one-paragraph bill summary. Under a bill from the council speaker, Julie Menin, it would be "unlawful to market, offer for sale, sell, or deploy an artificial intelligence (AI) model in New York City that has not received third-party validation or does not have a technical capability to be shut down by a human operator," according to the council's own record of the hearing. The penalty is a fixed $25,000 "per instance."

The bill was one of ten on the agenda of a Committee of the Whole, the full 51-member council sitting as one committee, which it had not done since 2022, TNW reports. Anthropic sent Logan Graham, head of its Frontier Red Team; OpenAI sent Morgan Dwyer, its head of policy development and operations; Alice Friend of Google and Shane Cahill of Meta appeared by video. Google, OpenAI and Anthropic agreed to attend only after the council warned them of subpoenas, TNW says. SpaceXAI, which the council did subpoena, sent no one, and Menin said she would ask a judge to enforce it.

Validation under Menin's bill would cover task performance, disparate impact, data privacy and safety, including whether the shut-down capability works. Validators would have to certify to the developer and to the city's Cyber Command whether a model is ready for deployment, and disclose any interest they hold in it. Cyber Command would write the rules, including who qualifies as a validator. A companion bill from Council Member Carl Wilson would require any advertisement for an AI model in the city to disclose whether it has been validated, and would bar materially false or misleading claims about a model's safety, with civil penalties of up to $25,000.

The idea is not foreign to the companies in the room. Anthropic's chief executive committed last month to give outside evaluators employee-level access, and Sam Altman said OpenAI would do the same. The accord six AI chiefs signed with the President last week asks each company for an outside auditor. Those are undertakings the companies chose. New York's version would make validation a condition of selling in the city, with qualifications set by a city agency. Neither the bill summary nor the bill text defines what counts as an "instance" for a model offered to millions of users; on our reading, that definition will decide whether $25,000 is a nuisance or a deterrent.

Menin pressed the four company witnesses on two questions, TNW reports. First, she asked each to put a number on the risk of a catastrophic worst case. OpenAI's Dwyer said the exact figure did not matter because no level of that risk was acceptable. "We should not train models that we cannot make an extremely strong case that we can keep under human control," Dwyer said. Menin called the answer "flippant at best". Friend said forecasting catastrophic risk "is not a perfect science at this stage". Second, Menin asked who carried insurance against catastrophic risks. None of the four raised a hand. Graham said Anthropic welcomes "smart regulation" and that state and local governments have a role; Friend backed a "comprehensive" federal framework.

The other witnesses were former insiders. "On the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction," said Jacob Coxon, a former OpenAI and Anthropic researcher who left Anthropic in September, according to TNW. PIX11 reported that he also told the council AI could bring "tremendous upsides" if it goes well. Daniel Kokotajlo and Alex Turner, formerly of OpenAI and Google DeepMind, testified remotely under subpoena. Council staff published a table of 13 publicly reported AI incidents in 2026 involving models from the four companies, and exhibits setting each company's public safety claims beside later incidents, TNW says.

The rest of the package targets the gaps those incidents exposed. A bill from Council Member Kamillah Hanks would require city contractors and agencies to report AI safety incidents to Cyber Command within 24 hours, and Cyber Command to disclose them publicly within 24 hours, a contrast with the 84 days it took for an OpenAI model's entry into an Australian government portal to be reported. Virginia Maloney's bill would let people sue model providers for foreseeable harm from third-party misuse where the company failed to put reasonable safeguards in place. Menin's second bill would pay anyone who reports an AI violation 25 per cent of what the city recovers, or 50 per cent if they bring the case themselves. Kevin Riley's would extend the city's whistleblower law to AI; others cover chatbot privacy, AI's effect on city jobs, AI emergency planning and deepfakes of officials and candidates.

None of it has passed. Every bill on the agenda was laid over by the committee, the council's record shows, and PIX11 reported that members expect to take them up on Thursday. Menin has framed the effort as filling a federal gap; when the bills were announced in September she cited Washington's "failure" to enact meaningful rules, 6sqft reported, while saying: "NYC is fast becoming the technology and AI capital of the world, which we want to encourage." Coxon told the council measures like these "may be helpful in the short term", TNW reported. The labs have said they want outside checks. New York's council is proposing to put a price on deploying without them.