If you are working in AI safety, while presenting information to interested non-AI safety or transitioning personnel, avoid portraying high values of p(doom) without sharing counterarguments. In addition, please give them time and point them towards emotional support so the unprecedented risk can be processed at their own pace.
Encourage those transitioning and working in the field to find ways to deal with said p(doom) looming over their heads on a day-to-day basis. This work will most likely require exploration and lead to highly individual outcomes.
Share and celebrate progress in the field! Managing one's own motivation is difficult work and reminding each other that achievements/progress are being made can be helpful.
Intro
I have found myself overwhelmed when thinking about existential risk. Joking with peers about it provided necessary momentary distraction. Usually, after a moment of relief, discussions and work could continue more productively. I would like to make the case that repeatedly presenting catastrophic outcomes without emotional support and/or time to process is only partially helpful, both for attracting and retaining talent in the field.
My aim in this post is to verbalise my impressions around how existential risk, its urgency and the sensation of "doom" in particular are presented and circulated within the field. If you consider yourself ASI-pilled, the below perspective could seem outright wrong. In that case, please help me understand your viewpoint without getting overwhelmed by a sense of hopelessness.
Basis
I assume that many readers here will have engaged with the podcast with Spencer Greenberg. I think it is a very good baseline and find myself agreeing with his statements and recommendations, in particular about finding a version of yourself that is productive without being in a constant state of hypervigilance and anxiety.
However, in my opinion, how existential risk is being communicated and processed adds an additional strain that goes beyond general career advice, like finding engaging work, and frameworks like positive vs. negative stress.
- Allow some time to process: Decoupling p(doom) and corresponding shortening timelines, especially felt after the HuggingFace incident, from a perceived need to respond immediately in the moment. The field is moving fast and hard to catch up as it is. I believe dropping started work and escalating to more drastic measures primarily helps regain a sense of control and may not necessarily be the most impactful or correct choice.
- Vary content: In particular non-grim media and communications, like a recent YouTube video on why AI could take us back to the Middle Ages, which resulted out of a collaboration with 80,000 Hours exactly because there was a felt need for non-grim material. I personally enjoyed meme competitions at courses and bootcamps.
- Provide emotional support: That can be just space for talking about the emotional work this includes. Even if the provider is not AI-safety pilled, free resources like overcome.org.uk, where one-on-one video calls are being offered, do exist and of course there are wider, existing summaries even on this very platform regarding available resources. Having said that, I believe that mental health concerns due to fears of x-risks materialising are not so unique that they require interventions specific to AI safety.
- Acknowledge progress: Provide tangible goals and celebrate wins. I can't shake off the feeling that even if PauseAI were to achieve their goals, the attention and anxiety would immediately shift to the next thing before people had celebrated that success.
Counterarguments
- The constant reminder of p(doom) helps me produce better work. Response: Great, if that is your experience. I argue that providing space for people to take AI-safety risk very seriously, work productively and create impact, without feeling hopelessness all the time, could increase the pool of potential participants. How I try to do that on a day-to-day basis: Trying to find a positive topic that provides some energy, e.g. digitally via some hopescrolling, or the classic touch some grass advice. This will obviously vary massively on an individual basis.
- Logical clarity is more important to me than a false sense of comfort. My brain is just wired that way. Response: I would primarily ask for the benefit of the doubt. People may independently of the factual correctness need some help understanding the proposed argument. They might require more time to process or regulate themselves before they can share their counterarguments and resume the discussion. Personal grim-o-meters can vary and explicit communication of your personal setting could help us understand each other.
- Finally, if the belief is that existential risk requires dramatic changes in one's life and not acting as such creates enormous stress due to a lack of perceived integrity, then I would like to gently ask whether heroism or self-sacrifice can be done on a smaller scale or in a sustainable manner. If the individual answer to that is no, then I do acknowledge that there is real tension. However, expecting talent to just increase resilience can be perceived as very demanding and ultimately contributes to keeping the AI safety tent small, contrary to the field's explicit call for additional support.
Closing
I would like to finish by emphasising that people are not islands. Even if you were willing to take very drastic measures like quitting your job or moving countries, a case could be made that your wellbeing correlates with that of those around you.
tl;dr
Intro
I have found myself overwhelmed when thinking about existential risk. Joking with peers about it provided necessary momentary distraction. Usually, after a moment of relief, discussions and work could continue more productively. I would like to make the case that repeatedly presenting catastrophic outcomes without emotional support and/or time to process is only partially helpful, both for attracting and retaining talent in the field.
My aim in this post is to verbalise my impressions around how existential risk, its urgency and the sensation of "doom" in particular are presented and circulated within the field. If you consider yourself ASI-pilled, the below perspective could seem outright wrong. In that case, please help me understand your viewpoint without getting overwhelmed by a sense of hopelessness.
Basis
I assume that many readers here will have engaged with the podcast with Spencer Greenberg. I think it is a very good baseline and find myself agreeing with his statements and recommendations, in particular about finding a version of yourself that is productive without being in a constant state of hypervigilance and anxiety.
However, in my opinion, how existential risk is being communicated and processed adds an additional strain that goes beyond general career advice, like finding engaging work, and frameworks like positive vs. negative stress.
Being confronted with the estimate by the Alignment Science lead at Anthropic that there is a >10% chance that all humans will be killed by AI, which was shared in response to a pre-training researcher resigning due to safety concerns, without extensive further explanation is an extreme input. And our brain needs time to absorb and digest it. The fact that experts in the field share vastly different estimates, effectively ranging from <0.01% to 99.9%, with differing definitions and timeframes, doesn't really help reduce initial anxiety.
Recommendations
What would I have preferred?
- Allow some time to process: Decoupling p(doom) and corresponding shortening timelines, especially felt after the HuggingFace incident, from a perceived need to respond immediately in the moment. The field is moving fast and hard to catch up as it is. I believe dropping started work and escalating to more drastic measures primarily helps regain a sense of control and may not necessarily be the most impactful or correct choice.
- Vary content: In particular non-grim media and communications, like a recent YouTube video on why AI could take us back to the Middle Ages, which resulted out of a collaboration with 80,000 Hours exactly because there was a felt need for non-grim material. I personally enjoyed meme competitions at courses and bootcamps.
- Provide emotional support: That can be just space for talking about the emotional work this includes. Even if the provider is not AI-safety pilled, free resources like overcome.org.uk, where one-on-one video calls are being offered, do exist and of course there are wider, existing summaries even on this very platform regarding available resources. Having said that, I believe that mental health concerns due to fears of x-risks materialising are not so unique that they require interventions specific to AI safety.
- Acknowledge progress: Provide tangible goals and celebrate wins. I can't shake off the feeling that even if PauseAI were to achieve their goals, the attention and anxiety would immediately shift to the next thing before people had celebrated that success.
Counterarguments
- The constant reminder of p(doom) helps me produce better work. Response: Great, if that is your experience. I argue that providing space for people to take AI-safety risk very seriously, work productively and create impact, without feeling hopelessness all the time, could increase the pool of potential participants. How I try to do that on a day-to-day basis: Trying to find a positive topic that provides some energy, e.g. digitally via some hopescrolling, or the classic touch some grass advice. This will obviously vary massively on an individual basis.
- Logical clarity is more important to me than a false sense of comfort. My brain is just wired that way. Response: I would primarily ask for the benefit of the doubt. People may independently of the factual correctness need some help understanding the proposed argument. They might require more time to process or regulate themselves before they can share their counterarguments and resume the discussion. Personal grim-o-meters can vary and explicit communication of your personal setting could help us understand each other.
- Finally, if the belief is that existential risk requires dramatic changes in one's life and not acting as such creates enormous stress due to a lack of perceived integrity, then I would like to gently ask whether heroism or self-sacrifice can be done on a smaller scale or in a sustainable manner. If the individual answer to that is no, then I do acknowledge that there is real tension. However, expecting talent to just increase resilience can be perceived as very demanding and ultimately contributes to keeping the AI safety tent small, contrary to the field's explicit call for additional support.
Closing
I would like to finish by emphasising that people are not islands. Even if you were willing to take very drastic measures like quitting your job or moving countries, a case could be made that your wellbeing correlates with that of those around you.