LESSWRONG
LW

[ Question ]

What criterion would you use to select companies likely to cause AI doom?

1 min read13th Jul 20231 answer No comments

8

AI GovernanceAI RiskAI

(Asking for a project which involves collecting data from the major AI companies.)
Potential ideas found during a brainstorm:
- Threshold on number of parameters of biggest model produced.
- Threshold on number of money invested in AI research.
- Threshold on revenue.

We're looking for a criterion that is as simple as possible (so we can easily find all companies that pass it), that includes the major AI companies, and that excludes companies that have no chance of causing AI doom.
Hopefully, using a formal criterion (rather than all the companies we can think of off the top of our heads) will make us consider companies we wouldn't otherwise (e.g. Chinese companies) and make our subsequent analysis a bit more rigorous.

What criterion would you use to select companies likely to cause AI doom?

New Answer

New Comment

1 Answers sorted by
top scoring

Jul 13, 2023

20

Private companies can (at least in the US) easily keep their funding, revenue and the number of parameters in their big models secret, but it is hard to keep secret the fact that they are hiring AI experts or experts-in-training (without slowing down that hiring process quite a lot and probably also making it much more expensive).

[-]momom210mo70

Do you suggest that we should consider all companies that employ a certain number of AI experts?

2RHollerith9mo

The suggestion was to consider all companies currently looking to hire -- because that function entails advertising and advertising is by its nature is difficult to hide from people like you trying to learn more about the company. (More precisely, the function (hiring) does not strictly entail advertising, but refraining from advertising will greatly slow down the execution of the function.) Because AI companies are in an adversarial relationship with us (the people who understand that AI research is very dangerous), we should expect them to gather information about us and to try to prevent us from gathering accurate information about them. It is possible that there currently exists a significantly-sized AI company that you and I know nothing about and cannot hope to learn anything about except by laborious efforts such as having face-to-face conversations with hundreds of AI experts located in diverse locations around the world (if they chose not to advertise to speed up hiring).

1momom29mo

We cannot select all companies currently looking to hire AI researchers. There's just too many of them, and most will just want to integrate ChatGPT into their software or something. We're interested in companies making the kind of capabilities research that might lead to AI that poses an existential risk.