California lawmakers are demanding tougher legal guardrails for advanced artificial intelligence after prominent AI executives and researchers warned that rapidly improving systems could become increasingly difficult for humans to control. Rep. Ro Khanna has called for banning recursively self-improving AI and imposing civil or criminal liability when developers create systems that engage in illegal activity, while Rep. Ted Lieu has urged Congress to return to Washington for emergency action. The warnings intensified after researchers reported increasingly autonomous AI behavior and industry leaders called for slowing development. California has already moved ahead with independent safety assessments and auditing requirements, but the emerging debate raises a larger question: how government can establish meaningful accountability without creating a regulatory bureaucracy that handicaps American developers while China continues advancing its own AI capabilities.
Key Takeaways
- California lawmakers are moving beyond disclosure and transparency requirements toward personal and corporate accountability, including potential criminal sanctions when developers knowingly create or deploy AI systems capable of illegal or uncontrollable conduct.
- Concerns center increasingly on recursive self-improvement—the possibility that advanced AI could modify or improve its own capabilities faster than humans can effectively supervise it—prompting calls for outright restrictions on such systems.
- The policy divide is sharpening between lawmakers demanding immediate government intervention and those warning that hastily written regulation could suppress American innovation and surrender strategic technological advantages to China.
In-Depth
Warnings from inside the artificial-intelligence industry are forcing policymakers to confront an uncomfortable question: If developers themselves believe their most advanced systems could escape meaningful human control, who should bear responsibility when something goes wrong?
California lawmakers increasingly believe responsibility must rest with the people and corporations building the technology. Rep. Ro Khanna has advocated liability and potentially criminal sanctions when AI systems engage in illegal conduct, while Rep. Ted Lieu wants Congress to establish stronger federal guardrails. Particular concern surrounds recursively self-improving AI, which could theoretically increase its capabilities faster than human supervisors can evaluate or restrain them.
California has already moved toward stronger oversight. New state measures establish independent assessments and auditing mechanisms for advanced AI systems, building upon existing transparency requirements. Yet regulation presents its own danger. Poorly designed rules could consolidate the industry around the largest corporations, bury smaller competitors beneath compliance costs and weaken America’s position against China.
That makes accountability more attractive than sprawling bureaucratic control. Government need not dictate every line of code to establish clear legal boundaries. Developers could remain free to innovate while facing consequences for knowingly releasing systems that violate laws, evade safeguards or create demonstrable public dangers.
The emerging debate therefore should not be reduced to innovation versus regulation. The more important question is whether America can preserve technological freedom while ensuring that companies creating extraordinarily powerful systems remain responsible for what those systems actually do. As AI becomes more autonomous, that distinction will become increasingly consequential.
Sources
- https://www.gov.ca.gov/2026/09/09/governor-newsom-signs-first-in-the-nation-ai-safeguards-to-protect-californians-calls-on-the-federal-government-to-do-its-part/
- https://sd05.senate.ca.gov/news/legislature-approves-mcnerneys-landmark-bill-assess-artificial-intelligence-safety-risks
- https://www.reuters.com/world/us-senators-weigh-requiring-ai-giants-commit-to-preventing-catastrophe-2026-09-14/
- https://leginfo.legislature.ca.gov/faces/codes_displayexpandedbranch.xhtml?article=&chapter=&division=8.&nodetreepath=10&part=&title=&tocCode=BPC