There is a question about AI regulation that I keep coming back to.
If the companies building the world’s most capable AI systems genuinely believe those systems could become dangerous enough to threaten cybersecurity, biological security, critical infrastructure, or even our ability to control them, why are those same companies racing to build more capable models?
And if they believe the risks are serious enough to require government intervention, why should we assume the rules they recommend are economically neutral?
Those questions do not require a conspiracy theory.
They require something much less exotic: an understanding of incentives.
OpenAI and Anthropic are commercial organizations competing for customers, capital, compute, talent, distribution, and technological leadership. They also employ people who appear sincerely concerned about AI safety. Their public policy materials warn about cybersecurity, biological misuse, and possible loss-of-control risks while arguing for stronger government institutions and oversight.
Both things can be true at once.
A company can believe that a technology creates genuine risks while also preferring a regulatory structure that works to its strategic advantage.
That is the part of the debate that deserves more attention.
The most useful question is not simply whether AI should be regulated. AI already interacts with laws governing consumers, civil rights, finance, health, safety, fraud, and other consequential activities, and federal regulators have emphasized that existing legal protections do not disappear because AI is involved.
The harder questions are:
- Who gets to define “safe”?
- Who can afford to prove compliance?
- Who supplies technical expertise to regulators?
- What happens to open-weight competitors?
- Does federal regulation replace state law?
- When an AI system causes harm, does regulatory compliance strengthen accountability or become a shield against it?
Once you ask those questions, the AI regulation debate looks very different.
Big AI Really Did Ask to Be Regulated
Let’s start with what is not speculative.
In May 2023, OpenAI CEO Sam Altman appeared before the U.S. Senate and argued for government oversight of increasingly capable AI.
OpenAI’s subsequent written answers to the Senate discussed possible licensing or registration mechanisms for sufficiently capable systems, external testing and validation, burdens on smaller firms, and how existing or new liability frameworks might apply.
That was not a one-time position.
By June 2026, OpenAI’s public-policy agenda supported frontier-safety frameworks that include transparency, catastrophic-risk evaluations, whistleblower protections, and enforceable developer accountability.
OpenAI also advocated a comprehensive federal framework and argued that, once such a framework exists, federal law should preempt state laws addressing the same frontier-safety risks.
Anthropic has gone at least as far.
Its 2026 Advanced AI Framework proposes obligations for frontier developers, disclosure of risks and incidents, independent evaluation, civil enforcement, and government authority capable of blocking or deterring dangerous deployments.
Taken at face value, these are not companies asking to be left entirely alone.
So why would they support regulation?
One coherent answer is that unilateral restraint does not solve a collective problem.
Imagine OpenAI slowing development while Anthropic, Google, Meta, Chinese laboratories, or a future competitor continues. OpenAI would absorb the competitive cost while the technology, and any associated systemic risk, continued advancing elsewhere.
Anthropic acknowledges versions of this collective-action problem in its Responsible Scaling Policy, arguing that some advanced safeguards may eventually become difficult or impossible for one company to implement unilaterally and may require broader industry or government action.
So the question, “If they think AI might be dangerous, why don’t they simply stop building it?” has a rational answer.
They may believe stopping alone accomplishes very little.
But that leads to a second issue.
The companies asking government to constrain the industry are also among the organizations best positioned to explain to government what those constraints should look like.
That is where the incentives become important.
Follow the Incentives: Safety, Certainty, and the Regulatory Moat
Regulation creates costs.
But those costs do not affect every competitor equally.
A frontier lab with enormous financial resources can hire policy teams, lawyers, safety researchers, red teams, security specialists, compliance professionals, and outside evaluators. It can build reporting systems and prepare for audits.
A startup may face many of the same fixed requirements with a fraction of those resources.
That is the basic mechanism behind what critics call an AI regulatory moat: compliance obligations that are manageable for incumbents but expensive enough to increase barriers to entry for challengers.
We do not have to speculate about whether this mechanism can exist.
The companies themselves have acknowledged it.
In an August 2025 letter to California Governor Gavin Newsom, OpenAI argued that smaller developers should not face compliance burdens designed for much larger companies. Its reasoning was straightforward: large firms are better positioned to absorb those costs than early-stage teams.
Anthropic has made the concern even more explicit.
In a policy paper on third-party AI testing, Anthropic warned that evaluation systems that are difficult or expensive to administer can favor companies with greater resources. It also discussed regulatory capture and the risk that high fixed compliance costs could advantage larger businesses.
That does not prove that Anthropic or OpenAI is intentionally trying to create a competitive moat.
It establishes something more useful: the mechanism is credible enough that the companies themselves have warned about it.
What Regulatory Capture Actually Means
Academic research points toward the same structural problem.
A peer-reviewed 2024 study on regulatory capture in general-purpose AI governance used interviews with 17 AI-policy experts and identified concerns about industry influence through agenda-setting, advocacy, information management, academic relationships, status, and media influence.
Regulatory capture does not simply mean “a lot of regulation.”
It means regulation or enforcement becomes excessively responsive to the interests of the regulated industry.
That can take different forms.
It might mean costly rules that disproportionately burden new entrants.
It could mean weak regulation.
It could involve favorable definitions, enforcement priorities, exemptions, or liability provisions.
And none of those outcomes requires coordinated bad faith.
Capture can emerge because regulators need information they do not possess. Industry may have more technical expertise than government or civil society. Large companies can devote more personnel and money to the policy process. Policymakers may also naturally turn to the largest and most visible firms when trying to understand a rapidly changing technology.
AI Companies Are Increasing Their Policy Presence
AI companies are also spending more to participate in that process.
OpenSecrets data reported by TechCrunch showed OpenAI’s federal lobbying expenditures increasing from approximately $260,000 in 2023 to $1.76 million in 2024, while Anthropic’s increased from roughly $280,000 to $720,000.
Those are not extraordinary sums by Washington standards.
The direction matters more.
As AI regulation became increasingly important commercially and strategically, frontier AI companies expanded their participation in the policy process.
Lobbying is not evidence of corruption. Companies have a legitimate interest in advocating for policies that affect their businesses.
It is also reasonable to recognize that regulation can produce several benefits for the companies subject to it.
A company can favor regulation because it improves safety.
It can favor one federal standard because complying with a single framework may be easier than navigating many state regimes.
It can favor regulation because enterprise customers value certification and predictable rules reduce investment risk.
Compliance requirements can also raise competitors’ costs.
Several motives can exist at the same time.
That is why the fact that “Big AI wants regulation” tells us relatively little by itself.
The important question is which regulation it wants.
The Details of AI Regulation Matter More Than the Speeches
OpenAI’s evolving position illustrates the point.
It opposed California’s SB 1047 in 2024, arguing that frontier-model regulation should primarily be addressed at the federal level.
By 2026, OpenAI was supporting newer state frameworks while presenting state convergence as a path toward eventual comprehensive federal regulation.
Anthropic took a different approach to SB 1047 and later supported California’s SB 53, which Governor Gavin Newsom signed in September 2025.
These are not identical policy agendas.
That distinction is useful.
The relevant story is not simply that AI companies favor regulation.
They have positions on:
- What regulation should cover
- Which developers should be subject to it
- Where capability thresholds should sit
- Who should conduct evaluations
- Who should enforce the rules
- Whether federal law should preempt state requirements
- What liability should remain after compliance
For businesses evaluating AI regulatory capture and AI compliance, those details matter more than broad statements about being “pro-regulation” or “anti-regulation.”
Open Weights, Competition, and the Regulatory Moat
The open-model question makes the regulatory-moat theory both stronger and more complicated.
Open-weight AI matters because it can reduce dependence on a handful of hosted-model providers.
Organizations can download model weights, run them on their own infrastructure, adapt them, and in many cases avoid paying a frontier-model provider for every inference.
“Open weight” is not necessarily identical to “open source” in the traditional software sense. Licenses, source code, training recipes, datasets, and documentation can differ substantially.
But economically, widely available weights can reduce barriers to experimentation and deployment.
The U.S. Commerce Department’s NTIA Open Model Weights Report concluded that foundation models with widely available weights can broaden participation by less-resourced actors and decentralize AI market control away from a small number of large developers.
NTIA also documented genuine concerns involving national security, safety, privacy, civil rights, and accountability.
Based on the available evidence, however, it did not conclude that restrictions on open weights were warranted and instead recommended continued monitoring and evidence gathering.
That makes open weights more than a philosophical debate about developer freedom.
They are also a competition issue.
A July 2026 industry letter on open weights and American AI leadership similarly argued that open-weight models can expand access, strengthen competition, and reduce dependence on a small number of providers.
So does Big AI simply want to regulate open models out of existence?
The evidence does not support that blanket conclusion.
OpenAI released gpt-oss-120b and gpt-oss-20b as open-weight models under an Apache 2.0 license in August 2025 and argued that open models can lower barriers for smaller organizations and resource-constrained sectors.
Anthropic, despite taking one of the industry’s stronger public positions on catastrophic frontier risks, stated in July 2026 that it does not support a categorical ban on open-weight models. Its public position on open weights describes non-dangerous open-weight models as valuable and recognizes their role in competition, while advocating capability-based safety testing for sufficiently powerful systems.
That is important counterevidence.
But it does not eliminate the competition question.
It changes the questions we need to ask.
Where Are the Thresholds Drawn?
Which models trigger mandatory evaluations?
Who pays for them?
What qualifies as a catastrophic capability?
Can an independent open-model laboratory satisfy the same documentation, cybersecurity, and reporting requirements as a multibillion-dollar frontier company?
Are smaller developers exempt?
Does regulation attach to:
- Training compute?
- Model capability?
- Company revenue?
- Deployment?
- Downstream use?
Those details determine whether frontier AI regulation functions primarily as a risk-control system or also creates significant barriers to entry.
Anthropic’s 2026 framework attempts to address this by targeting highly capable systems and larger developers through capability and financial thresholds rather than applying the same obligations to every AI developer.
China Adds Another Layer to the Debate
Geopolitical competition makes AI regulation even more complicated.
OpenAI’s proposals for the U.S. AI Action Plan explicitly framed domestic AI policy in the context of competition with China.
OpenAI argued that overly burdensome domestic regulation could benefit Chinese competitors and advocated a combination of federal policy, infrastructure expansion, export controls, and measures intended to preserve U.S. AI leadership.
Anthropic similarly treats China as a major strategic concern but emphasizes controls on advanced chips and chipmaking equipment, preventing industrial-scale model distillation, and evaluating dangerous capabilities rather than simply banning open-weight models.
This creates a genuine policy tension.
Frontier AI may become powerful enough that governments want stronger oversight.
At the same time, U.S. policymakers and companies are concerned about slowing domestic development while strategic competitors continue advancing.
Those positions are not logically incompatible.
But together they create strong incentives to design regulation that reduces certain risks without materially weakening the competitive position of the companies or countries adopting the rules.
That is why businesses should pay attention to the design of AI safety regulation, not simply its stated purpose.
Liability: Who Pays When AI Causes Harm?
There is another issue that may ultimately matter more than licensing.
Who pays when AI actually causes harm?
Across much of the economy, deploying software does not automatically eliminate organizational responsibility.
Knight Capital: Software Does Not Remove Accountability
Consider Knight Capital.
In 2012, a defective software deployment caused Knight’s automated trading system to send more than four million orders into U.S. markets during roughly 45 minutes while attempting to execute only 212 customer orders.
Knight accumulated billions of dollars in unintended positions and lost more than $460 million.
The SEC’s enforcement action focused on inadequate safeguards and control failures, including automated error emails generated before the market opened that were not acted upon.
The regulator examined the company, its deployment processes, testing, monitoring, safeguards, and controls.
AI is not automated securities trading.
But the narrower accountability principle is useful:
“The software did it” is not a complete answer.
Existing Law Already Applies to Automated Systems
We are already seeing that principle applied to algorithmic systems.
The U.S. Department of Justice’s housing-discrimination case against Meta concerned algorithms used in housing advertising. The resulting settlement required changes to Meta’s advertising system under court oversight.
The FTC and other federal agencies have likewise emphasized that existing legal protections continue to apply when automated systems are involved.
A particularly simple example comes from Canada.
In Moffatt v. Air Canada, the airline’s chatbot supplied incorrect information about bereavement fares. Air Canada attempted to distinguish the chatbot’s information from other material on its website.
The tribunal rejected that distinction and held the airline responsible for the information delivered through its own automated system.
The financial stakes were modest.
The accountability principle was not.
The company chose to deploy the system.
The chatbot did not become a separate legal entity that absorbed the company’s responsibility.
Generative AI Is Creating Harder Liability Questions
U.S. courts are now confronting more difficult versions of the issue.
In Garcia v. Character Technologies, a federal judge in May 2025 allowed significant claims involving alleged harms connected to a Character.AI chatbot to survive motions to dismiss.
The ruling was procedural rather than a final determination of liability.
But it illustrates the questions courts are beginning to address around negligence, product liability, consumer protection, and constitutional doctrines applied to generative AI.
That makes the liability language in future federal AI legislation especially important.
Does compliance with a federal safety framework become evidence that a company exercised reasonable care?
Does federal law preempt state tort claims?
Does compliance create a safe harbor?
Can an injured consumer still sue?
Does the regulator become the primary authority deciding what qualifies as safe?
Anthropic’s 2026 framework addresses some of these questions directly. It argues for narrow federal preemption and says compliance with a federal regime should not automatically create immunity, a safe harbor, or a presumption against liability under otherwise applicable state law.
That is meaningful evidence against the strongest claim that calls for AI regulation are simply an effort to escape litigation.
OpenAI’s current policy agenda also advocates developer accountability while favoring federal preemption of state rules covering the same frontier-safety risks once a comprehensive federal framework exists.
The eventual legislative wording would matter enormously.
The Other Accountability Problem: Regulator Dependence
Governments cannot effectively regulate extremely complex frontier systems without technical expertise.
But much of that expertise currently resides inside the companies being regulated.
The models may be proprietary.
Evaluation techniques are still developing.
Compute is expensive.
Internal safety information may be confidential.
Regulators may lack direct access to the infrastructure needed to independently reproduce company findings.
That creates an information asymmetry.
And versions of this institutional problem have appeared in other industries.
The Lesson From Boeing Is About Oversight Structure
After the Boeing 737 MAX crashes, the U.S. Department of Transportation’s inspector general examined FAA certification and delegation.
Its review of the 737 MAX certification process identified communication and oversight gaps, including concerns around the FAA’s understanding of Boeing’s safety assessments and the independence of delegated authorization.
AI is not aviation.
OpenAI is not Boeing.
The useful comparison is narrower:
When a regulator depends heavily on a regulated company for technical judgment, independence must be deliberately built into the oversight system.
Interestingly, Anthropic makes a similar argument.
Its proposal for third-party testing calls for stronger independent evaluation capacity partly because an industry-led ecosystem can favor well-resourced companies and increase regulatory-capture risk.
That should be treated as an important design consideration.
If governments create new frontier-AI oversight institutions, those institutions need sufficient personnel, technical infrastructure, model access, compute, legal authority, and budget to independently evaluate the companies they regulate.
Otherwise, the risk is that oversight becomes too dependent on industry judgment.
The Right Test for AI Regulation
So, are large AI companies asking for regulation because their leaders genuinely worry about what frontier models may eventually do?
They may be.
Is regulation commercially useful to established firms?
It can be.
Could both things be true?
Absolutely.
That is why AI regulation should not be evaluated only by how strict it sounds.
It should be evaluated by the incentives and institutional structures it creates.
Ask who defines the safety standard.
Ask who gets to change it.
Ask whether obligations scale with actual capability and organizational size.
Ask who can afford the audits.
Ask whether evaluators are genuinely independent.
Ask whether regulators can reproduce company findings with their own technical resources.
Ask what happens to open-weight AI models.
Ask whether state consumer-protection and tort remedies survive.
Ask whether regulatory compliance creates new liability protections.
Ask who benefits from federal preemption.
And ask whether the rules preserve meaningful competition or unintentionally entrench the companies already best equipped to comply with them.
The central principle should be straightforward:
The company that builds, deploys, controls, and profits from an AI system should not be able to transfer responsibility for that system entirely to government oversight.
Regulation can create minimum standards.
It can require:
- Testing
- Disclosures
- Cybersecurity controls
- Incident reporting
- Independent evaluations
- Corrective action
It can establish consequences when companies ignore known risks.
What it should not become is a substitute for accountability.
That is the frontier-model paradox worth watching.
The same companies warning that this technology may become extraordinarily powerful are racing to make it more powerful while participating heavily in the debate over what rules should govern that race.
The answer is not to assume bad faith.
It is to design AI regulation with the assumption that every participant, including companies, governments, competitors, and regulators, operates under incentives.
Because they do.
Frequently Asked Questions About AI Regulation
Why Do AI Companies Want Regulation?
AI companies publicly cite safety, national security, regulatory predictability, public trust, and the need for common standards.
Regulation can also create economic benefits. A uniform federal regime may reduce state-by-state complexity, while high fixed compliance costs can affect smaller competitors more heavily than large incumbents.
OpenAI and Anthropic have themselves acknowledged this difference in compliance capacity.
What Is an AI Regulatory Moat?
An AI regulatory moat is a competitive barrier created when licensing, audits, evaluations, reporting, cybersecurity requirements, or other fixed compliance costs are substantially easier for large incumbents to absorb than for startups or smaller open-model developers.
A regulatory moat is an economic effect.
Its existence does not automatically prove intentional regulatory capture.
What Is AI Regulatory Capture?
Regulatory capture occurs when regulation or enforcement becomes excessively aligned with the interests of the regulated industry rather than the broader public interest.
AI-specific peer-reviewed research has identified agenda-setting, advocacy, information management, and disparities in technical expertise as possible channels for industry influence.
Are AI Companies Liable When AI Causes Harm?
Potentially.
Liability depends on jurisdiction, facts, and legal theory.
Existing consumer-protection, civil-rights, negligence, and other laws can apply to automated systems, while courts are beginning to decide how product-liability and related doctrines apply specifically to generative AI.
Garcia v. Character Technologies is one important early example, although the 2025 ruling discussed above was procedural rather than a final determination of liability.
Does AI Regulation Threaten Open-Source or Open-Weight AI?
It can if compliance requirements are structured around costs or controls that only large providers can realistically satisfy.
But the current evidence is more complicated than a simple closed-versus-open conflict.
NTIA has identified competitive benefits from widely available model weights, OpenAI publishes open-weight models, and Anthropic says it opposes a categorical open-weight ban while supporting safety evaluation of sufficiently capable models.
Why Does China Matter to U.S. AI Regulation?
Leading U.S. AI companies increasingly frame regulation as both a safety issue and a geopolitical competition issue.
OpenAI has argued that overly burdensome domestic regulation can benefit Chinese competitors, while Anthropic emphasizes controls on advanced chips, industrial-scale model distillation, and dangerous capabilities.
That creates pressure to design rules that address AI risk without materially weakening U.S. technological competitiveness.
