Ideas for improving epistemics in AI safety outreach
In 2022 and 2023, there has been a growing focus on recruiting talented individuals to work on mitigating the potential existential risks posed by artificial intelligence. For example, we’ve seen an increase in the number of university clubs, retreats, and workshops dedicated to introducing people to the issue of existential risk from AI. However, these efforts might foster an environment with suboptimal epistemics. Given the goal of enabling people to contribute positively to AI safety, there’s an incentive to focus on that without worrying as much about whether our arguments are solid. Many people working on field building are not domain experts in AI safety or machine learning but are motivated due to a belief that AI safety is an important issue. Some participants may hold the belief that addressing the risks associated with AI is important, without fully comprehending their reasoning behind this belief or having engaged with strong counterarguments. This post is a brief examination of this issue and suggests some ideas to improve epistemics in outreach efforts. Note: I first drafted this in December 2022. Since then, concern about AI x-risk has been increasingly discussed in the mainstream, so AI safety field builders should hopefully be using fewer weird, epistemically poor arguments. Still, I think epistemics are still relevant to discuss after a recent post noted poor epistemics in EA community building. What are some ways that AI safety field building may be epistemically unhealthy? * Organizers may promote arguments for AI safety that may be (comparatively*) compelling yet flawed * Advancing arguments promoting the importance of AI safety while neglecting opposing arguments * E.g., citing that x% of researchers believe that AI has an y% chance of causing an existential catastrophe, without the caveat that experts have widely differing views * Confidently making arguments that are flawed or have insufficiently justified premises * E.g.
I guess this should wait for the final draft of the GPAI Code of Practice to be released?