It seems to have become common practice for discussions around AI governance on lesswrong to name specific AI labs such as Facebook AI research, OpenAI and DeepMind. Or top researchers such Demis Hassabis or Yann LeCun.

Wouldn't it be better to use a placeholder such as "AI lab" without referring to a specific AI lab?


  1. Less antagonistic. (MAIN ADVANTAGE)

For instance, quoting from the recent post on list of lethalities:

restricting yourself to doing X will not prevent Facebook AI Research from destroying the world six months later

Compare this to:

restricting yourself to doing X will not prevent another AI lab from destroying the world six months later

A researcher working at Facebook AI research today is less likely to parse the latter in an adversarial or emotional way. Even if they disagree with the content, it is more likely that they will disagree in a manner that allows them to change their mind later on, because they perceive it less adversarially.

Even if you know that "AI lab" is also referring to you or the lab you work for or run, you might process it as less jarring versus reading your own first name or your lab's name in the same para.

As someone working at such a lab, you may also feel more confident that the authors of such posts are not trying to incite crowds to take action against you, including extremist ones, and that they're willing to resolve disagreements politely.

  1. Technically more accurate

We don't actually know which AI lab or even which country will build AGI (aligned or not), to such a high likelihood that we can name one lab today.


  1. Social pressure is needed

You might believe social pressure and calling labs by name is more likely to get them to appreciate the seriousness and stakes involved, or take responsibility.

I'm personally not convinced this helps, on net, but it doesn't seem crazy for someone to believe Bad Cop is more effective than Good Cop. Keen on thoughts.

  1. Harder to parse

Content of the post may be easier to parse with specific examples. This seems like a minor disadvantage relative to the other points.

Also, one can have a disclaimer post to link to which clearly spells out what "another AI lab" means.


17 comments, sorted by Click to highlight new comments since: Today at 10:46 PM
New Comment

For instance, quoting from the recent post on list of lethalities:

restricting yourself to doing X will not prevent Facebook AI Research from destroying the world six months later

Compare this to:

restricting yourself to doing X will not prevent another AI lab from destroying the world six months later

One problem with your suggested reformulation is that they do not mean the same thing. Naming FAIR conveys information above and beyond 'another AI lab', which additional information would be hard to sum up, much less put into an equally short phrase: there's reasons it's 'FAIR' which is used in that sentence about catch-up competitors & unilateralist curse, and why various other AI lab names which one might think could be dropped in like 'DeepMind' or 'Sberbank' or 'MILA' or 'BAIR' or 'OpenAI' or 'Nvidia' or 'MSR' would all be wrong in various ways*, but it's hard to even explain. Similarly, there are things like the Google Brain vs DeepMind rivalry which affect how scaling research has been playing out, for the worse, and are relevant to safety, but it would be absurd to try to talk about it in a generic sense or use pseudonyms.

(To try to explain anyway: basically, they're kind of a perpetual second-mover second-best player with a bit of a chip on their shoulder from poaching and scooping - consider Darkforest - with little commitment to AI safety either in the real AI risk sense or AI bias sense and mostly in denial that there is any problem (eg. LeCun: "There is no such thing as Artificial General Intelligence because there is no such thing as General Intelligence."), and get looked down on for being part of Facebook with overly-applied focus, seen as the most ethically-compromised and amoral part of FANG eager to ship tech because of Zuckerberg's conviction that more tech & transparency is always better (at least as long as it increases FB growth), and a historical backstory where FB struggled to hire DL people to begin with despite Zuckerberg strongly believing in DL potential from almost AlexNet, being spurned by the likes of Hassabis, settling for LeCun/Pesenti/etc, because despite access to huge resources & data & industrial application FB/Zuckerberg were already widely hated. This description takes up a whole paragraph, and you probably still don't know a good chunk of what I'm referring to with Darkforest or Zuckerberg NIPS shopping, and if you disagreed, asking why this doesn't characterize OA or GB or Baidu equally well, that would be another set of paragraphs, and so on. But if you already know this, as you should if you are going to discuss these things, then use of 'FAIR' immediately conjures the family of plausible complex-failure disaster scenarios where eg. DM or another top player succeeds in not-instantly-failed AGI, it gets controlled by the DM safety board overruling any commercial pressures, only to trigger Zuck into ordering a replication effort to leapfrog the over-cautious 'dumb fucks' at DM, about which there is zero internal mechanism or oversight to oppose the CEO/controlling shareholder himself nor anyone who cares, which succeeds because the mere fact of AGI success will tend to indicate what the solution is, but then fails.)

* This is one reason I invested so much effort in link-icons on, incidentally. Research papers are not published in isolation, and there are lots of 'flavors' and long-standing projects and interests. Much like an expert in a field can often predict who wrote a paper despite blinding (ex ungue leonem), you can often tell which lab wrote a paper even with all affiliation information scrubbed, because they bring with them so much infrastructure, approaches, preferences, and interests. I can predict a remarkable amount of, say, the Gato paper simply from following Decision Transformer research + the title + 'DeepMind' rather than 'Google' or 'OpenAI' or 'BAIR'. (For a natural experiment demonstrating lab flavor, simply compare Gato to its fraternal twin paper, GB's Multi-Game Transformer.) So I find it very useful to have a super-quick icon denoting a Facebook vs OpenAI vs DM vs Nvidia etc paper. You the ordinary reader may not find such annotations that useful, yet, but you will if you git gud.

Thanks, this is a great point!

For those who have such inside views on the differences between AI labs and think it is meaningful to discuss them, I think they should discuss them. (Should they discuss it in public or private? Maybe public is okay, I don't have a strong opinion.)

For those who do not have such inside views and do not have additional information content to convey beyond just "another AI lab" I think my original question is still a valid one.

I think it's good that someone is bringing this up. I think as a community we want to be deliberate and thoughtful with this class of things.

That being said, my read is that the main failure mode with advocacy at the moment isn't "capabilities researchers are having emotional responses to being called out which is making it hard for them to engage seriously with x-risk."
It's "they literally have no idea that anyone thinks what they are doing is bad."

Consider FAIR trying their hardest to open-source capabilities work with OPT. The tone and content of the responses shows overwhelming support for doing something that is, in my worldview, really, really bad.

I would feel much better if these people at least glanced their eyeballs over arguments for not open-source capabilities. Using the names of specific labs surely makes it more likely that the relevant writing ends up in front of them?

Fair point. I don't know if your take on failure mode is true or not, but it seems a reasonable opinion to have.

As for hoping they read stuff, there might be more effective ways to get content out to them than just posting publicly and hoping it catches their eye. But I can't deny that is also a strategy.

Yeh so thinking a little more I'm not sure my original comment conveyed everything I was hoping to. I'll add that even if you could get a side of A4 explaining AI x-risk in front of a capabilities researcher at <big_capabilities_lab>, I think they would be much more likely to engage with it if <big_capabilities_lab> is mentioned.

I think arguments will probably be more salient if they include "and you personally, intentionally or not, are entangled with this."

Saying that, I don't have any data about the above. I'm keen to hear any personal experiences anyone else might have in this area.

I've done quite a bit of thinking about this, and I'm pretty familiar with the area.

If a corporation has a brand, and you have no idea how powerful, aggressive, or exploitative that corporation is (e.g. Facebook, Disney, etc), then it's best not to write anything that calls out that brand. If you go on Reddit and write something publically about how awful Dr. Pepper is, then you're entangling yourself into the ongoing conflict between Coca Cola and Pepsi, whether you know about it or not. And if you don't know what you're getting into, or even aren't sure, then you certainly aren't prepared to model the potential consequences.

Ok not sure I understand this. Are you saying "Big corps are both powerful and complicated. Trying to model their response is intractably difficult so under that uncertainty you are better to just steer clear?"

Yes, that's a very good way of putting it. I will be more careful to think about inferential distance from now on.

Facebook AI labs: I wonder what people are saying about us on the internet? Hopefully nothing, obviously. But lets look, because we can.

Facebook AI Labs: *inundated with randomly ordered yudkowsky paragraphs with no context whatsoever*

Facebook AI Labs: Well, at least the problem is contained to a single set of internet forums, and it looks crazy to us, but stranger things have spread to tens of millions of people. We'd better go ask someone if they think these people should be taken care of.



Facebook AI labs: Wait, these are actuals NGOs? And they're saying bad things about AI and Facebook AI Labs? Well, that's clearly a problem to bring to management.

This seems like a case of making a rule to fix a problem that doesn't exist.

Are people harassing individual AI labs or researchers? The tendency for reasonable people who are worried about AI safety should be to not do so, since it predictably won't help the cause and can hurt. So far there does not seem to be any such problem of harassment discernible from background noise.

Naming individual labs and / or researchers is interesting, useful, and keeps things "real." 

So, to be clear, you don't think confidently naming people by first name as destroying the world can be parsed emotionally by them?

It's harder to focus on object-level when your name is on something imo, whether or not it's targeted at you or intended to be harassing.

So, to be clear, you don't think confidently naming people by first name as destroying the world can be parsed emotionally by them?

Mentions of AI companies / AI personalities on LW will intrinsically tend to be adversarial, even if the author spares a polemic or use of terms like "so and so is working to destroy the world" because misaligned AI destroying the world is clearly THE focus of this community. So it can be argued that to be meaningful, a policy of no names would need to be applied to practically any discussion of AI as even if some AI content is framed positively by the author, the community at large will predictably tend to see it in existential risk terms.

That's one issue. Personally, the calculus seems pretty simple: this well-behaved community and its concerns are largely not taken seriously by "the powers" who will predictably create AGI, there is little sign that these concerns will be taken seriously before reaching AGI, and there is almost no reason to think that humanity will pause to take a break and think "maybe we should put this on hold since we've made no discernible progress toward any alignment solutions" before someone trains and runs an AGI. So a conclusion that could be drawn from this is, we might as well have nice uncensored talks about AI free from petty rules until then.

re: first para

Yes that is true, I think you've weakened my opinion a bit.

re: second para

Are you just saying it is so difficult to convince capabilities researchers to stop that we shouldn't even bother to try?

Anyone can try, this seems way out in a practically invisible part of the tail of obstacles to not being destroyed by AGI, if it's even an obstacle at all. 


I think I just disagree. LW and alignment forum content will be likely be noticed by many people working on capabilities, whether or not they agree with it, especially as capabilities progress further.

Just as an example, Sam Altman tweeted out Yudkowsky's latest doom post.

And convincing capabilities researchers seems like potentially the biggest obstacle, not a small one.

I haven't justified my beliefs much, but I can if you can point out where exactly we're mostly likely to disagree.

Less antagonist.

Should be "less antagonistic", unless you're talking about the bad guy in a story.