Introduction: I was recently reading Orienting to 1 year AGI timelines and was surprised to see that, at least according to Nikola Jurkovic, there are only 15 FTE working on AI Safety x CA State legislation! At the same time I’m somewhat already involved in state politics, so this seems like potentially a high value place to get involved. This is my effort to do research on the space and figure out what my next steps are.
Current laws:
SB 53 requires frontier developers to publish safety frameworks and report critical incidents, and protects whistleblowers. Recommended reading: What SB 53, California’s new AI law, does
SB 813 (McNerney) creates a voluntary tier of government-credentialed auditors called Independent Verification Organizations (IVOs). Doing an audit under the law is “relevant to, but not conclusive of” liability if the company is later sued over AI harm. Nothing requires labs to be audited. IVO designation criteria is due May 1, 2027.
AB 1405 (Bauer-Kahan) creates a mandatory registry for anyone who conducts AI audits, with independence and reporting standards, starting in 2029. It doesn’t require developers to be audited.
Executive Order N-9-26 (September 18, 2026) Newsom directed state agencies to accelerate implementation of the audit laws and develop recommendations on mandatory independent verification, embedded auditors, kill switches, and expanded incident reporting. The recommendations were due November 16, 2026 (That’s today! I’ll post something when they get released to the public)
In conclusion, so far it's just SB 53. Ordinarily California bills usually take effect the following January, so anything on a 6-12 month clock needs an urgency clause (a two-thirds vote), a budget vehicle, an executive order, or emergency powers. That means getting new bills passed in time to make a difference on short time lines is much harder, although not impossible.
Priorities:
Kill switch / emergency shutdown authority: SB 1047, a 2024 bill, would have required developers of the most powerful models to maintain the ability to promptly shut them down, along with safety testing and annual third-party audits. It passed both chambers, but was vetoed by Gavin Newsom. AI labs tend to oppose anything that gives the government the ability to shut down models of their own purgative, but I currently expected a version of this to be passed even without further intervention. Priority: medium.
Transparency requirements: Current transparency obligations are primarily developer-authored disclosures. As we've seen, inside risk reports consistently under estimates threats and fail to disclose important information (such as rogues instances). Forcing labs to accept more independent outside evaluators is high priority.
Binding safety requirements: Nothing sets a substantive standard a frontier model must meet before release. Any binding requirement would also face federal pushback: DOJ’s AI Litigation Taskforce targets California’s laws, among others. There's no consensus around what "pacing the frontier" looks like, but targeted safety requirements are among the more popular options. A well written law could very likely get the labs support. Priority: high.
Whistleblower protection: SB 53 already gives relatively good protection for whistleblowers. There are undoubtedly many ways it could be improved, but currently the bill seems good enough and is a low priority for revisiting/strengthening.
What are the other important areas?
Current people working in the space:
Encode AI is a youth-led advocacy nonprofit and one of the main groups behind California’s frontier AI laws. It came up earlier as the group that has asked for enforced minimum safety standards before release. "Pro-human, pro-AI."
Fathom "is an independent nonprofit whose mission is to build a global architecture that helps society navigate the transition to a world with AI by fostering trust, safety, and innovation."
Note: everyone now cares about AI, I'm focusing on org's with a real focus on X risks, and not child safety or any other issue. They must also have people actively working on shaping CA policy, and not just endorsing bills. There must be more in this category that I don't know of, please tell me .
What needs to happen in the next 6 to 12 months?
Overall I was surprised by how proactive and effective the state government has been relative to California's usual track record at tackling complex issues, sadly this is not a case where merely above average is sufficient.
Get safety requirements implemented before labs can publish new models/train more powerful models. I'm sure someone is currently working on writing this legislation (or at least I hope several people are), but I currently don't know who.
Improving access for third party auditors.
Pass a new kill switch act when Xavier Becerra takes office, ideally under an emergency clause so that we don't have to wait for 2028 for it to come into effect.
Postlude: On a personal note, I’m looking for any collaborators, connections, funding or jobs that can help move towards this goal. I’m also currently working on a letter writing campaign with a friend and we are very open to suggestions!
Introduction: I was recently reading Orienting to 1 year AGI timelines and was surprised to see that, at least according to Nikola Jurkovic, there are only 15 FTE working on AI Safety x CA State legislation! At the same time I’m somewhat already involved in state politics, so this seems like potentially a high value place to get involved. This is my effort to do research on the space and figure out what my next steps are.
Current laws:
Executive Order N-9-26 (September 18, 2026) Newsom directed state agencies to accelerate implementation of the audit laws and develop recommendations on mandatory independent verification, embedded auditors, kill switches, and expanded incident reporting. The recommendations were due November 16, 2026 (That’s today! I’ll post something when they get released to the public)
In conclusion, so far it's just SB 53. Ordinarily California bills usually take effect the following January, so anything on a 6-12 month clock needs an urgency clause (a two-thirds vote), a budget vehicle, an executive order, or emergency powers. That means getting new bills passed in time to make a difference on short time lines is much harder, although not impossible.
Priorities:
Current people working in the space:
Note: everyone now cares about AI, I'm focusing on org's with a real focus on X risks, and not child safety or any other issue. They must also have people actively working on shaping CA policy, and not just endorsing bills. There must be more in this category that I don't know of, please tell me .
What needs to happen in the next 6 to 12 months?
Overall I was surprised by how proactive and effective the state government has been relative to California's usual track record at tackling complex issues, sadly this is not a case where merely above average is sufficient.
Postlude: On a personal note, I’m looking for any collaborators, connections, funding or jobs that can help move towards this goal. I’m also currently working on a letter writing campaign with a friend and we are very open to suggestions!