Idea. People in the outsider orgs should talk directly to lab employees, and explain the arguments for model access parity which are most legible/persuasive to them. They could try to get commitments from the labs, but I don’t expect these to be binding, so outsiders should mostly focus on getting insiders to actually believe the arguments.

Overall judgement. This looks pretty good, but it's difficult to centralise or to front-load. I do think that, on the margin, people should be writing short memos of the form "Why labs should endorse our work?"

Concrete steps:

1. Talk to outsiders who have already negotiated with the labs for model access, to understand how this goes — what access did they have, how did they get it, what did they have to concede, where do negotiations typically stall? Most of this isn't written down; ask people who've done it.
2. Talk to a small number of sympathetic lab employees who could actually push this internally. Ask the lab insiders what the cruxes/bottlenecks are in their internal conversations, and help alleviate those. Maybe you should write a 2-pager optimised for forwarding (this would be similar to a grant proposal, but asking for model access rather than money).
3. You probably want to frame model access parity as a productivity issue, i.e. "Look how much value our work provides you — don't you want to make us more productive?" This seems to have worked reasonable well in the pre-deployment testing space. If your organisation has use cases which aren't in the labs' interests, then they could potentially piggyback on the work which the labs like. (Although I'm worried this won't work for activities that are actively hostile to labs.)
4. Try to snowball more support, e.g. use your existing lab connections to book meetings with more lab connections. Try to generally increase the knowledge within the lab about what your org does and why.
5. Probably you'll need to understand the concerns of less-sympathetic lab employees who could veto the model access, e.g. the security teams.
6. Plausibly, if the model access gap is sufficiently high, then orgs should send someone to the labs specifically so they can champion the orgs internally.

Pros:

1. At least currently, lab employees seem best positioned to push for model access parity. Historically, when outsider orgs have been given special access to models or artefacts, this has routed via individual lab employees, or small teams.
2. This requires spending some social and professional capital, but not much.

Cons:

1. I’m worried that lab leadership might mislead the sympathetic insiders, i.e. sympathetic insiders tell the outsiders “Don’t worry, lab leadership said this will happen” and then it doesn’t happen.
2. The labs might worry about looking biased by providing special access to their friends. They might think "If we give org X the latest model, then we'd have to give it to org Y as well, otherwise we look biased". This is especially worrying if Org X isn't legible to the the public, investors, or governments — e.g. they work on esoteric philosophy.
3. It might distort the incentives of the outsiders if they are relying on goodwill with the labs to do their work.
4. This doesn’t work for orgs which need to be actively hostile to labs, i.e. orgs trying to generate evidence that the labs are being reckless.

Idea. Outsiders could make public requests for model access parity. The hope is that someone would be persuaded to help make this happen, e.g. policymakers, public, etc.

Overall judgement. I think this should probably happen concurrently with advocacy to lab employees. It seems somewhat stickier than relying on the lab's goodwill. My main worry is probably that the work of outsiders might not be legible to the groups with leverage over the labs.

Concrete steps:

1. Outsiders could talk more about model access parity on podcasts, blogs, etc. They discuss how crucial model access would be for their work, and why their work is important.
2. Potentially, outsiders could sign an open letter, probably with prestigious academics and the top outsider orgs. (This seems a little overblown at this stage.)
3. We probably want to frame this as a checks-and-balances issue, rather than a productivity issue. Why? Because it's probably politically feasible for labs to claim that it's not worth uplifting outsiders, given the security risks. Or labs might give access to stooges who won't challenge them, or chumps who will be incompetent at challenging them even if they try. However, if our framing is “checks-and-balances” then the public/policymakers will defer less to labs about model access decisions.
4. We could creatively interpret commitments that labs have already made as requiring model access parity, e.g. "You've already agreed to third-party evaluations. These evaluations are ineffective, because we don't have enough time. But we could do our job effectively if we had more uplift." See Appendix 3.4-5 of the EU AI Act Code of Practice.^[3]

Pros:

1. USG is powerful enough to force labs to do things strongly against their interest. (And for UK and EU, weakly against their interest).
2. Governments might be a blocker on model access parity, not the labs. See the recent Mythos saga. Hence, it might be important to reverse the government’s position.
3. The “checks-and-balances” argument is probably more accurate than the “let us help the labs” argument. That is, I think outsiders will have highest leverage for activities which are weakly against the lab's interests, where direct advocacy might not be enough.
4. I don't think it will be difficult to explain to policymakers that model access parity would make outsider orgs 50x more effecient. Since Mythos, I think they understand that AI can make certain activities much more efficient.

Cons:

1. If labs don’t want to provide access, then I think they could have reasonable-sounding counterarguments, which would be difficult to disprove from the outside (e.g. “misuse risks are too high”)
2. Project Glasswing was legible to policymakers, because they could understand why cyberdefence is important. But some of the outsider efforts are less legible, e.g. speculative futurism, automated philosophy, etc.
3. Outsiders might be doing stuff which the government doesn't like, e.g. slowing down labs (if the political mood is that this is bad), or stopping power concentration.

Idea. We could push labs to have an internal policy on outsider access. The hope is to have some transparency about how these decisions are actually made. They are free to revise the policy at any time, but not silently.

Overall judgement. I like this. If we want to ensure model access parity, then this would help us track our progress — we can see how much our interventions seem to improve the policy. My main worry is that this forces the lab to choose a policy they can defend, which might be worse than the policy they would actually want to follow, because the policy is forced seem impartial/unbiased.

Concrete steps:

1. Talk to lab employees about how model access decisions are actually made. Cross-refence this against what outsider orgs think. Check if there are disagreements and try to resolve them. Try to understand what the implicit policy is currently.
2. Ask sympathetic lab employees to push internally on making the policu explicit. This would probably need to be signed off by leadership, the security teams, and other groups.
3. Encourage labs to publish their policies.
4. Perhaps help labs coordinate on multilaterally expanding the policies, e.g. each frontier lab commits to providing METR with the best internal models to help them audit their competitors' models. This might be a coordination problem, i.e. the labs would do this multilaterally but not unilaterally.

Pros:

1. Transparency around model access policies might lead to a race-to-the-top. Potential hires would be more excited about working at the lab which was seen as supporting outsider orgs.
2. This makes the status quo legible and makes regression visible.
3. This is an easier request than a specific policy.
4. Once the policy exists, it might become sticky internally. When new models come out, labs will follow the policy by default, rather than renegotiating with every outsider org. So it might be possible for external AI safety researchers to get and keep their access.
5. To the extent that we can rely on these policies, it helps the outsiders track whether the model access gap will grow or shrink.
6. It helps focus our advocacy efforts. We can say "Please change this line in the document to this other line”. Without a policy, we don’t know what we actually want them to change.

Cons:

1. The labs might choose a bad policy and anchor on that.
2. It’s probably not in the labs interest to tie their hands, and we don’t have much leverage to get labs to tie their hands.
3. The policy we actually want is something like “orgs trying to make the future go well” — but this isn’t an objective, easily-adjudicated policy, which can easily be defended to the public.

Idea. We could start an org decidated to model access parity. It would do whatever was necessary to ensure model access parity, including:

1. Negotiating with labs on behalf of third-party researchers.
2. Knowing the policies of each labs, and who to talk to in each lab.
3. Helping outsider orgs alleviate bottlenecks on model access — e.g. if the bottleneck is security, then the DRI org might hire a security consultancy for their client.
4. They might work with regulators, or the people advocating to policymakers.
5. Generally digging through any schlep to make model access happen

Overall judgement. I think this is pretty good, but might be overkill at this stage. I can see an org like this becoming a top priority. If Generator starts pumping out generalists, then this seems like a good option for them.

Concrete steps:

1. Find a skilled generalist, pair them with funding.
2. They would talk to outsider orgs who need model access, and see how much interest there would be in an org which helped them. If there is enough interest, then they would start an org and begin some of the activities.
3. I imagine they would mostly be muddling through myopically, trying to ensure that their client orgs have the model access that they say they need.

Pros:

1. A dedicated organisation would change the conversation from "labs vs. many loosely-coordinated organisations" to "labs vs. one professional negotiator."
2. Their would be to amortise expertise and experience across the core outsiders, reducing redundancy.
3. They would build connections with the labs and streamline the process.
4. Small orgs or independent researchers would know to contact them.

Cons:

1. Maybe the staff at the outsider orgs should do the negotiations themselves, because they have the social and professional connections with the lab employees.
2. To do this well, the dedicated org would need to be staffed with people with high opportunity cost.

Idea. An AI governance researcher would draft a model access policy. Labs could then adopt this, or we could push governments to enforce this.

Overall judgement. This is probably worthwhile for an AI governance researcher who felt motivated to do this. But it’s plausible that a bad version of this would be counterproductive.

Concrete steps:

1. The researcher would talk with outsiders to find what polices would be most attractive for them. And talk to lab insiders about which policies they would tolerate.
2. They would then choose some compromise between the labs and the outsiders, based on the political will.
3. We could then push this policy on the labs, using direct advocacy, or lobbying the government, or using other leverage.

Pros:

1. I think it would encourage labs to adopt their own model access policy, because they have a template from which to start with. Lab employees can ask themselves "What changes to this policy would we need before we would agree?"
2. If we have a good policy, this might anchor the discussion.
3. This seems like a well-scoped project, which won't take much FTE.

Cons:

1. We might sell ourselves short, e.g. write a policy which is weaker than what the labs would've given us.

Idea. Outsider orgs should think about why the labs (or regulators) might hesitate to provide the model access, and address those bottlenecks preemptively. This probably involves improving security. It might involve other things, as those bottlenecks become apparent.

Overall judgement. This doesn’t look attractive to me, because the bottlenecks will be so sensitive to the strategic landscape. My best guess is that we should resolve bottlenecks as they arise. If there are low-hanging fruits, then sure.

Concrete steps:

1. Talk to lab employees about what they expect the bottlenecks to be.
2. The current bottleneck is probably be security — so maybe hire a security consultancy, or get a certification, or pass the lab's security audit.
3. Maybe you should try to gain access to artefacts which aren't publicly available (e.g. hidden CoT traces, finetuning access). That will show you what the currrent bottlenecks are, and the future bottlenecks might be pretty similiar.

Pros:

1. When I spoke to senior lab employees about model access parity, this was the intervention they thought was best.
2. It's somewhat predicatable that security will be a bottleneck.
3. If resolving the bottlenecks requires a long lead-time, then you haven't got much choice other than pre-emption.

Cons:

1. It’s not clear what kinds of security the labs care about.
2. This will probably be costly, and impose friction on the outsider orgs.
3. Labs aren’t asking for this loudly. They haven't said "At some point, we will only deploy our models to organisations who have done bla."
4. Many outsiders already have good security, because they handle the pre-deployed models (e.g. METR, UK AISI, etc). But to my knowledge they can't use the best internal models to accelerate their research.
5. We might not see a significant model access gap for 6 months, and best security practices in 6 months might look different (because of AI progress in both offence and defence).
6. I'm hopeful that outsider orgs can muddle through if this becomes an issue, especially if the labs explicitly tell outsiders how to overcome the bottlenecks, e.g. the labs help the outsiders pass the security audit.

Idea. If outsiders have compute, then they can use this as a chip when they negotiate for model access. We can say to labs "You can rent our $1B neocloud, but only if you provide your best internal models to us".

Overall judgement. I think this is the stickiest intervention. But it's already being looked into by the relevant people.

Concrete steps:

1. Buy a datacentre. It might be important to physically own the GPUs and hire your own physical security. (I'm somewhat worried about compute contracts, because they might be renegged.)

Pros.

1. It seems robustly good if we have stuff we can trade with the labs — not just for model access parity, but for other things we want the labs to do.
2. The compute would be useful for other things, e.g. running experiments, renting to make money, etc.

Cons.

1. It costs a lot of money.
2. It seems like a big operational endeavour.
3. The labs might be blocked by regulation from providing model access, in which case, having compute you can offer the labs isn’t useful.

Idea. This proposal comes from Ryan Greenblatt.

Pseudo-employees. Make a class of pseudo AI company employees who don't have equity and are pretty separate.

Overall judgement: I think this looks good. My guess is that lab security teams might have an issue with this, but not an insumountable one. I don't know what concrete steps we could take now to make this more likely.

Concrete steps:

1. Third-party risk assessors might push to join the labs as pseudo-employees, e.g. internal model access, corporate laptop, slack access, physical access, etc. The goal is to make this a position with some precedence within the labs.
2. Governments might place CAISI staff into the labs.

Pros:

1. I think it'd be good if there were more partial insiders, i.e. non-employees with many of the attributes of full employees.
2. From what I understand, there are precedents in the banking industry, where regulators are based full-time in the main office of the institution.
3. It would overcome many of the potential bottlenecks, e.g. security concerns.

Cons:

1. There's probably a low upper limit on how many pseudo-employees a lab could have, e.g. 5% of the workforce. Anthropic has ~5000 employees, so this would be only a hundred or so pseudo-employees.
2. You could probably be a pseudo-employee of at most one lab.
3. You would probably need to sign aggressive NDAs.

Idea. This is conceptually similar to the idea of “Buy compute”. However, instead of using compute to trade with the labs, we use artefacts — such as datasets, techniques, etc. Currently, outsiders give this to labs for free, because they would prefer the labs had access to those artefacts than not to have them, all-things-considered. But if the labs would also prefer to have those artefacts, then it seems fair to ask for something in return, e.g. model access.

Overall judgement. I think the weaker version of this idea (below) might be good. The extreme version seems pretty terrible.

Concrete steps:

1. Extreme version: Outsiders continue doing work that aligns with the lab's incentives (e.g. improve safety tech, building benchmarks, etc). But we don't give this to the labs. We keep it secret, so we can trade this with the labs during crunch time in return for model access.
2. Weak version: Outsiders collaborate with the labs on work that aligns with the lab's incentives (e.g. improve safety tech, building benchmarks, etc). But we push hard for model access for the entire org. The model access can be used not only for the projects the lab commissioned, but also for projects the lab didn't commission.

Pros.

1. It seems cheaper than buying a datacentre.

Cons.

1. This can easily backfire. If we overestimate how highly the labs value the artefacts, then we might end up demanding too much from them, so they don't "buy" the artefacts. This could be worse than offering the artefacts for "free".
2. It will be difficult to credibly demonstrate to the labs that the artefacts are useful, before they have seen them. Although if the outsider org has a great track record, this might be less worrying.
3. It requires coordination among the outsiders, to collectively "go on strike".

Idea. This proposal comes from Daniel Kokotajlo:

Mandate equal access. If any of your employees have access to a model, the public must also have access to that model via an API. (the idea here is to prevent secret intelligence explosions, and also to make it very obvious that an intelligence explosion is happening when it happens & have lots of info about the details of it, the model shenanigans, etc. public., and also preventing concentration of power in a single AI company.)

Overall judgement. My guess is that Kokotajlo's policy would be better than the status quo. But it seems much less achievable than ensuring model access for third-parties.

Concrete steps:

1. I'm not sure what the concrete steps would be. Probably you would need to start a movement around this.
2. You would then draft some policies on this, and try to get them passed.

Pros

1. This request would be backed by a very broad coalition.
2. It would have other benefits, e.g. waking up the public, avoiding power concentrations.

Cons:

1. I think that labs currently think of outsider orgs as in-group. But if the outsiders formed this coalition with the rest of the world for model access, then the outsiders would be seen as more out-group.
2. If there was a big movement for public access, then labs might be more worried about giving access to the outsiders, because this would be seen as a concession to this movementl. The lab would worry about slippery slope effects.
3. It's unclear whether public access would even be good, because it increases misuse risk.
4. I think the interventions for ensuring public acess looks pretty different than the interventions for the outsider access.