Anthropic has laid out one of the most detailed Anthropic AI policy frameworks yet from a frontier lab, calling on governments to gain legal authority to block dangerous AI deployments, impose revenue-linked penalties on violators, and build public resilience against a new class of technological risks.
The company’s policy plan, released under what it calls its “Policy on the AI Exponential” approach, covers two broad areas: a technical and regulatory framework for the most powerful AI models, and an economic policy framework focused on how workers and society should share in AI’s financial gains. Together, the Anthropic AI policy frameworks represent a serious attempt to push policymakers toward rules that can keep pace with technology moving faster than most legislatures.
At its core, the proposal is an acknowledgment that current systems are not enough. Existing transparency laws in states like California and New York are welcomed, but Anthropic argues that public disclosure alone no longer matches the speed of frontier AI development. As a result, the company says something more structural — and more enforceable — is needed.
Summary
Anthropic proposes new AI policy frameworks for frontier model safety
Frontier model safety and economic preparation
The Advanced AI Framework focuses on the most capable systems in existence. Rather than applying broad rules across the entire industry, Anthropic draws a precise line: models trained above 10²⁵ floating-point operations would fall under the framework. So would companies generating more than $500 million in AI-related annual revenue, and firms spending more than $1 billion on AI research and development.
That scope matters. These thresholds exclude smaller developers and research labs while targeting the handful of organizations building models with genuinely transformative — and potentially dangerous — capabilities. It is a deliberate design choice aimed at avoiding regulatory overreach while addressing the systems that carry the highest risk profiles.
The economic preparation side of the proposal deals with workers facing disruption from automation, although the full details of labor protections remain less developed than the technical framework.
Government powers to block dangerous AI deployments
Perhaps the most significant element of the proposal is its call for governments to hold real authority over AI deployments. Anthropic wants policymakers to have the power to block or deter high-risk model releases before they reach the public — a level of oversight that does not currently exist in comprehensive form in the United States.
This is not a soft recommendation. The company envisions enforceable mechanisms backed by financial consequences. Civil penalties would be tied to global annual revenue, and repeat violations would carry escalating fines. The intent is to make non-compliance genuinely costly for the largest players in the industry.
Core safety, transparency and enforcement measures
Independent testing, safety documentation and risk reporting
Frontier developers would be required to test models before release and publish summaries, safety frameworks, and system cards documenting how those models behave. Regular risk reports describing the developer’s overall risk posture and safety work would also become mandatory.
This creates a paper trail. It means the public and regulators would have structured visibility into how companies assess their own systems, rather than relying on press releases or voluntary disclosures. Independent evaluators would review company-conducted tests and publish their own findings on model risk, adding a layer of external scrutiny that does not currently exist at scale.
Civil penalties tied to global revenue and repeat violations
The enforcement architecture is built around financial accountability. By tying civil penalties to global annual revenue rather than fixed dollar amounts, the framework aims to ensure that penalties actually sting for the largest AI companies, whose revenues can reach into the tens of billions.
Higher fines for repeat violations create additional deterrence. The message is straightforward: the first failure might be treated as a compliance gap, but continued violations signal something more deliberate, and the penalties should reflect that.
Security programs and model evaluations by independent experts
Beyond testing, companies would need to maintain strong security programs protecting model weights and training systems from both outside attackers and insider threats. Developers would describe their security programs publicly at a general level, with deeper details available to a designated government agency upon request.
The framework also calls on governments and industry to jointly set standards for independent evaluators — and to ensure those evaluators have the funding and access needed to review frontier models. That last point is harder than it sounds. Meaningful evaluation requires access to the systems companies are most protective of, which makes the selection and funding of independent evaluators a central implementation challenge.
Scope of regulation and the main AI risks Anthropic identified
Targeted models and companies by training scale and revenue
The 10²⁵ floating-point operations threshold and the $500 million revenue cutoff are not arbitrary. They reflect the practical reality that the most dangerous AI capabilities emerge at scale, from models trained on massive compute with enormous resource backing. Smaller models and smaller companies simply do not pose the same category of systemic risk.
This scoped approach makes the framework more politically defensible, more administratively practical, and less likely to stifle innovation across the broader industry. It also makes it easier to adjust as compute costs fall and capability thresholds shift over time.
Four main risk categories: biological, cyber, loss of control and automated research
Anthropic identifies four areas where advanced AI poses the most serious threats:
- Biological risk: Unsafe systems could assist in developing harmful viruses, even as the same AI tools support legitimate drug discovery.
- Cyber risk: Frontier models can identify serious software vulnerabilities at scale, raising direct concerns for hospitals, energy grids, and other critical infrastructure.
- Loss of control: Systems operating outside developer intentions could cause harms that are difficult to reverse or contain.
- Automated AI research: AI systems accelerating their own development could compound biological, cyber, and control risks if adequate safeguards are not in place.
The breadth of this list reflects how the company thinks about risk — not as isolated technical failures, but as interconnected threats that can amplify one another. An AI system that finds software vulnerabilities at scale, for instance, could also accelerate the development of biological threats if it operates without sufficient constraints.
Public resilience and the pace of AI governance
Biological and cybersecurity resilience measures
The second half of the framework shifts from regulation to preparedness. Anthropic recommends that governments build real-world buffers against AI-enabled harms, including gene synthesis screening, early-warning biosurveillance systems, protective equipment stockpiles, and tools to reduce airborne transmission of biological threats.
On the cyber side, the proposal calls for hardening internet infrastructure, supporting critical infrastructure operators, replacing legacy systems in essential services, and establishing a dedicated government function to track frontier cyber capabilities. These are not measures that can be implemented quickly, but the proposal frames them as essential groundwork for a world where the most powerful AI systems are widely deployed.
Anthropic acknowledges that work on loss-of-control and automated research risks is less mature, calling for better tools to detect, contain, or shut down unsafe systems as the science develops.
Why AI governance has to move faster
The underlying argument throughout the proposal is straightforward: AI capabilities are advancing faster than the governance structures designed to manage them. Anthropic wants policymakers to treat this gap as urgent — not as a long-term policy exercise, but as an immediate structural problem.
The company’s suggestion that regulators start with lighter rules and adjust them over time is a pragmatic concession to the difficulty of getting comprehensive legislation passed. It also reflects a concern about locking in frameworks that become obsolete as the technology evolves.
What makes this proposal analytically interesting is its origin. Anthropic is itself one of the frontier developers that would fall under these rules. Advocating for regulation that applies directly to its own products and revenue is either a calculated move to shape policy or a genuine belief that the risks are serious enough to warrant outside constraints on the industry — including on itself. The two possibilities are not mutually exclusive, and how policymakers interpret that ambiguity will likely shape how seriously they engage with what is, on its technical merits, one of the more substantive AI governance blueprints to emerge from the private sector.
FAQ
What triggers the government’s authority to block an AI deployment under Anthropic’s proposal?
Under the framework, governments would gain authority to block or deter AI deployments deemed high-risk, particularly those involving frontier models trained above 10²⁵ floating-point operations or developed by companies with significant AI revenue. The exact trigger conditions for intervention are not fully defined in the current proposal.
How are independent evaluators selected and funded according to the framework?
The proposal calls for governments and industry to jointly set standards for independent evaluators. It also says evaluators need both funding and direct access to frontier models, but the specific selection process and funding mechanisms are not yet detailed.
What are the biological risks associated with advanced AI models?
Anthropic warns that unsafe frontier AI systems could assist in developing harmful viruses. The same capabilities that raise these concerns also support beneficial applications like drug discovery, which makes the dual-use nature of biological AI risk particularly complex to manage.
How does the framework address cybersecurity for critical infrastructure?
The proposal recommends stronger internet software, support for operators of critical infrastructure, replacement of legacy systems in essential services, and a dedicated government function for tracking frontier cyber capabilities. It identifies hospitals and energy grids as key areas of concern.
What penalties apply for repeat violations of the proposed AI regulations?
Civil penalties would be tied to a company’s global annual revenue, and repeat violations would trigger higher fines. The framework is designed to ensure that penalties are financially significant relative to the scale of the largest AI companies.
