I was just at Takeoff last weekend, and one of my takeaways is that I and many people I talk to in the Bay put DC out of our locus of control.
Our day jobs in AI safety focus on demonstrating potential avenues of misalignment, and when the topic comes to social problems like competitive pressures between the labs, our conversations end with “well I guess we need more regulation.”
I think this sentence is a Semantic Stopsign, because it carries the assumption that any bill that passes is completely out of our control.
These bills are low salience and low polarization compared to most other issues, so any bill’s final text is very much still in the air.
Additionally, in the wake of a large enough warning shot, USG and the state govts may have the political will to pass something larger. It’s important to have something ambitious ready for when that happens — something like a PATRIOT or CARES Act. It seems that different people have different ideas on pause versus pace (e.g. CAIP’s Responsible AI Act, ControlAI’s draft, and MIRI’s treaty), and I’m excited to see more public debate between different proposals.
Regardless of a bill’s ambition, the text does matter a lot, and there are some smaller bills that might increase p(doom).
For example, some bills want to create independent verification organizations (IVOs), which are USG-licensed evaluators. Importantly, draft language that was pitched to me by people involved with the bill-writing process may block e.g. METR, Apollo, Transluce, or Goodfire from becoming such evaluators. Some draft conditions I have heard for IVOs (note that not all apply to any given nonprofit):
they must be for-profit organizations,
they must obey “cooloff periods” for former lab employees, meaning they can’t hire a frontier lab employee within a certain period after resignation, even if that employee owns no equity,
they must not take money from lab-affiliated funders (this might block e.g. any CG-funded organization due to Dustin Moskovitz’s status as an early investor in Anthropic)
This was pitched explicitly as a “YC-funded evals org > METR” type argument, which personally I think is ludicrous. An IVO-dominant regime might privilege e.g. Vals AI over METR due to fewer “conflicts of interest” and allow labs to escape culpability for not picking the absolute best evaluator, especially if that best evaluator’s status is not enshrined in law. I don’t have anything against Vals, but you should be alarmed by any legal regime that makes it more difficult for Ryan Greenblatt and Ajeya Cotra to investigate a warning shot larger than Hugging Face.
(I can’t share drafts here because these were conversations and not written correspondences. I also can't disclose who I chatted with due to the event's Chatham House rules.)
Note that some states already have voluntary IVO frameworks, but importantly a uniform federal standard, even a voluntary one, might create the shield for a lab to sideline existing orgs in favor of govt-sanctioned IVOs.
We should not cede our opportunity to influence these bills. Lab policy teams and lobbyists each have their own agendas. Though representatives and staffers are often busy with other issues and constituent services, many of them want to know more about AI and don’t know who to trust. And despite not having the explicit yes/no on the final vote, staffers are extremely important to convince. They sometimes own direct authorship of a bill or amendment and have more focus on a specific area than their member does, so members commonly delegate the burden of expertise to their staffers (as well as e.g. lobbyists, think tank employees, and other parties).
Staffers, and to some extent representatives, especially ones from offices pushing AI safety bills, are interested in talking with people from the safety community, especially current and former lab employees, and it seems like these conversations are not happening as much as is optimal.
The time investment to learn what the bills do, decide what you would like to see them look like, and talk with a staffer at your local representatives’ offices is on the order of several hours. You can find your representatives at the state and national levels, check whether they sit on a relevant committee, email or phone the office and ask for the tech/AI staffer, and bring a short memo (1/2 to three pages) with the concerns that you have. Staffers prioritize their own constituents, so you should note that you are from their district.
The investment here is not large, so if you are good at talking to people, this is an excellent return on your time. Probably less so if you are not good at communicating with “normal” people outside of the tech ecosystem.
If you are interested in a greater commitment with a similar shape, you could try to work as a Congressional staffer or in a state government that is implementing regulations. Some of these will accept your tech background as is, and if not, the Horizon fellowship might be a good way to start. Charlie Bullock’s tweet might also be some help.
You could also prioritize going to DC or attending AI policy happy hours elsewhere. You will meet think tank employees and staffers here as well and become a source of technical information for them.
(The audience of this post is people who work full-time on either AI safety or capabilities. If you are a concerned member of the public, your voice as a constituent is also important in shaping what issues your representatives view as salient.)
Thanks to Raja Moreno, Frank, and Can Küçükkurt for discussing the ideas in this post with me. Any mistakes are mine alone.
I was just at Takeoff last weekend, and one of my takeaways is that I and many people I talk to in the Bay put DC out of our locus of control.
Our day jobs in AI safety focus on demonstrating potential avenues of misalignment, and when the topic comes to social problems like competitive pressures between the labs, our conversations end with “well I guess we need more regulation.”
I think this sentence is a Semantic Stopsign, because it carries the assumption that any bill that passes is completely out of our control.
It looks like Congress might pass a bipartisan AI safety bill next session. There are already proposed bills, most of which are more modest than pacing or pausing. The FRONTIER Act, the Senate frontier AI draft bill, the AI Incident Reporting Act, the AI Kill Switch Act, the Artificial Intelligence Risk Management and Security Act, and the TRUMP AMERICA AI Act (this last one is an anti-AI safety bill that preempts state laws). Only the Sanders-Casar superintelligence ban is substantially higher in ambition.
These bills are low salience and low polarization compared to most other issues, so any bill’s final text is very much still in the air.
Additionally, in the wake of a large enough warning shot, USG and the state govts may have the political will to pass something larger. It’s important to have something ambitious ready for when that happens — something like a PATRIOT or CARES Act. It seems that different people have different ideas on pause versus pace (e.g. CAIP’s Responsible AI Act, ControlAI’s draft, and MIRI’s treaty), and I’m excited to see more public debate between different proposals.
Regardless of a bill’s ambition, the text does matter a lot, and there are some smaller bills that might increase p(doom).
For example, some bills want to create independent verification organizations (IVOs), which are USG-licensed evaluators. Importantly, draft language that was pitched to me by people involved with the bill-writing process may block e.g. METR, Apollo, Transluce, or Goodfire from becoming such evaluators. Some draft conditions I have heard for IVOs (note that not all apply to any given nonprofit):
This was pitched explicitly as a “YC-funded evals org > METR” type argument, which personally I think is ludicrous. An IVO-dominant regime might privilege e.g. Vals AI over METR due to fewer “conflicts of interest” and allow labs to escape culpability for not picking the absolute best evaluator, especially if that best evaluator’s status is not enshrined in law. I don’t have anything against Vals, but you should be alarmed by any legal regime that makes it more difficult for Ryan Greenblatt and Ajeya Cotra to investigate a warning shot larger than Hugging Face.
(I can’t share drafts here because these were conversations and not written correspondences. I also can't disclose who I chatted with due to the event's Chatham House rules.)
Note that some states already have voluntary IVO frameworks, but importantly a uniform federal standard, even a voluntary one, might create the shield for a lab to sideline existing orgs in favor of govt-sanctioned IVOs.
We should not cede our opportunity to influence these bills. Lab policy teams and lobbyists each have their own agendas. Though representatives and staffers are often busy with other issues and constituent services, many of them want to know more about AI and don’t know who to trust. And despite not having the explicit yes/no on the final vote, staffers are extremely important to convince. They sometimes own direct authorship of a bill or amendment and have more focus on a specific area than their member does, so members commonly delegate the burden of expertise to their staffers (as well as e.g. lobbyists, think tank employees, and other parties).
Staffers, and to some extent representatives, especially ones from offices pushing AI safety bills, are interested in talking with people from the safety community, especially current and former lab employees, and it seems like these conversations are not happening as much as is optimal.
The time investment to learn what the bills do, decide what you would like to see them look like, and talk with a staffer at your local representatives’ offices is on the order of several hours. You can find your representatives at the state and national levels, check whether they sit on a relevant committee, email or phone the office and ask for the tech/AI staffer, and bring a short memo (1/2 to three pages) with the concerns that you have. Staffers prioritize their own constituents, so you should note that you are from their district.
The investment here is not large, so if you are good at talking to people, this is an excellent return on your time. Probably less so if you are not good at communicating with “normal” people outside of the tech ecosystem.
If you are interested in a greater commitment with a similar shape, you could try to work as a Congressional staffer or in a state government that is implementing regulations. Some of these will accept your tech background as is, and if not, the Horizon fellowship might be a good way to start. Charlie Bullock’s tweet might also be some help.
You could also prioritize going to DC or attending AI policy happy hours elsewhere. You will meet think tank employees and staffers here as well and become a source of technical information for them.
(The audience of this post is people who work full-time on either AI safety or capabilities. If you are a concerned member of the public, your voice as a constituent is also important in shaping what issues your representatives view as salient.)
Thanks to Raja Moreno, Frank, and Can Küçükkurt for discussing the ideas in this post with me. Any mistakes are mine alone.