This is a special post for quick takes by Ariel_. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I think people working in AI Safety, or at least in AI Policy/governance, should not be Invested into AI.
I guess a small amount of exposure is fine, 10-20% of net worth max. I had mentioned this to people for a few years, and didnt realize its perhaps not obvious.
Given the upcoming IPO, I was surprised to hear someone talk about trying to buy into it. Concretely, I have currently some exposure via an ETF but less than 20% in pure AI stuff.
Reasoning?
A coordinated pause (or an AI incident also, but less central to the argument) might cause financial distruption. We dont want to be emotionally & financially attached to either outcome, even if in practice the decision is unlikely to be made by one person.
The expected value of getting rich from AI isnt that high. We arent money bottlenecked already, and unless you are an AI Lab employee, I dont think you can realistically contribute much to the pool
We think we are unaffected by money, but I wouldnt bet on it. Its hard enough to be anywhere near rational as is, lets not add additional tempting factors.
Possible counterarguments -
Maybe having more money individually is useful if you think you are well placed to invest in very specific unusual bets, that currently you cant invest into.
If gaining financial independence is useful, that could be a factor (e.g. for doing work that funders are not interested in funding)
Maybe you have a great safety net and dont care if your net worth drops by 20-50% during an incident
I basically think the counterarguments only rarely apply, and that you should have a very high bar for thinking they apply to you, and that you are immune to the conflicts of interest.