Anthropic Wrote Its Own AI Safety Bill, and Its Critics Say It Mostly Protects Anthropic From Washington

The AI company that built its reputation on warning about catastrophe has written its own federal safety bill, and the people who usually cheer it on are furious. Anthropic has been quietly pitching a confidential draft to lawmakers on Capitol Hill, Politico reported, according to three people familiar with the talks.

The complaint from safety researchers is simple. The draft would make it harder, not easier, for Washington to stop a dangerous frontier model, and it would wipe out key parts of the state safety laws Anthropic once supported.

The bill has not been introduced. It is a discussion draft, and Anthropic had never openly endorsed a federal bill before.

A very high bar for pulling the plug

Under the draft, federal authorities could seek to halt a model only when it poses a foreseeable, imminent threat of more than $1 billion in damages or the death or serious injury of more than 50 people. The threat also has to fit one of three scenarios: creating or releasing biological weapons, a model escaping its developer's control, or cyberattacks.

Independent evaluators would register with the Commerce Department and periodically assess frontier labs and their models. The labs can pay those evaluators, as long as the pay does not depend on what they find. An evaluator that spots an imminent risk on the list has 72 hours after finishing its review to tell Commerce.

Even then, Commerce cannot shut anything down on its own. It has to ask a federal judge in the District Court for the District of Columbia for an order. Every other federal department, agency and officer would be barred from suspending a model's development, deployment or internal use.

The draft also preempts state rules on public risk disclosures, independent evaluations and critical incident reporting. In return it would require labs to keep safety frameworks, publish transparency reports for new models, protect whistleblowers and report critical incidents to Commerce.

An Anthropic spokesperson said the bill reflects the Advanced AI Framework the company published in June and has been promoting to lawmakers since. "These are sensible policies we believe Congress should act on," the spokesperson said.

The critics see a company afraid of its own government

Anthropic previously backed state laws like California's SB 53, which requires safety frameworks and reporting serious incidents to the state within 15 days. One critic, Alex Bores, pointed out that the company's own June framework said Congress should not preempt state law unless it creates a federal regime that meets or exceeds the framework's strongest measures.

The reaction from the safety world, collected by Zvi Mowshowitz, was brutal. Mowshowitz called the draft "unacceptable" and wrote that Anthropic seems "more afraid that the government might enforce rules against Anthropic, than they are worried about the ability of the government to enforce rules against everyone." Ketan Ramakrishnan called it "astonishingly bad."

Nathan Calvin said he "found myself viscerally angry at multiple points reading this bill." He argued that passing it would leave the public less protected against catastrophic risk than current law, and flagged a provision that appears to stop states from enforcing laws or court orders that suspend a model.

Kelsey Piper put the motive bluntly. Anthropic, she wrote, seems "terrified of the executive branch having the ability to flexibly make them stop at the drop of a hat." She added that "the administration's treatment of Anthropic was genuinely outrageous." Piper noted the safety movement wants the power to shut down superintelligent model development, while Anthropic wants those powers narrowed.

That fear has a history. An appeals court recently let the Pentagon keep blacklisting Anthropic over Claude, and US export controls reportedly sidelined two advanced Anthropic models over the summer.

Not everyone was so harsh. Charlie Bullock gave Anthropic "some credit for actually proposing a pretty reasonable AI safety bill," which he said is more than other frontier labs have done. He still called it weaker than the FRONTIER Act on rulemaking and emergency shutdown authority, and said it is "kind of wild" that the auditors need no government license. Mowshowitz also credited the whistleblower and security provisions, the annual audits and the listing of automated AI research as a risk.

A crowded field in Congress

The draft lands among several rivals. Reps. Lori Trahan and Jay Obernolte's FRONTIER Act would authorize government intervention against risky models while overriding some state laws. In the Senate, Majority Leader John Thune has worked with Sens. Amy Klobuchar and Ted Cruz on catastrophic risk legislation, while Sen. Maria Cantwell and some safety advocates want stronger federal review. Rep. Greg Casar and Sen. Bernie Sanders have called for pausing advanced AI altogether.

All of this is happening as regulators lean harder on the labs. The same week, the FTC hit OpenAI and Anthropic with civil subpoenas seeking records on safety incidents. The open question now is whether lawmakers will accept a ceiling on government power written by the company it is meant to regulate.

Some offers on this page may be paid placements or contain affiliate links.