Legible vs. Illegible AI Safety Problems
Some AI safety problems are legible (obvious or understandable) to company leaders and government policymakers, implying they are unlikely to deploy or allow deployment of an AI while those problems remain open (i.e., appear unsolved according to the information they have access to). But some problems are illegible (obscure or hard to understand, or in a common cognitive blind spot), meaning there is a high risk that leaders and policymakers will decide to deploy or allow deployment even if they are not solved. (Of course, this is a spectrum, but I am simplifying it to a binary for ease of exposition.) From an x-risk perspective, working on highly legible safety problems has low or even negative expected value. Similar to working on AI capabilities, it brings forward the date by which AGI/ASI will be deployed, leaving less time to solve the illegible x-safety problems. In contrast, working on the illegible problems (including by trying to make them more legible) does not have this issue and therefore has a much higher expected value (all else being equal, such as tractability). Note that according to this logic, success in making an illegible problem highly legible is almost as good as solving it! Problems that are illegible to leaders and policymakers are also more likely to be illegible to researchers and funders, and hence neglected. I think these considerations have been implicitly or intuitively driving my prioritization of problems to work on, but only appeared in my conscious, explicit reasoning today. (The idea/argument popped into my head upon waking up today. I think my brain was trying to figure out why I felt inexplicably bad upon hearing that Joe Carlsmith was joining Anthropic to work on alignment, despite repeatedly saying that I wanted to see more philosophers working on AI alignment/x-safety. I now realize what I really wanted was for philosophers, and more people in general, to work on the currently illegible problems, especially or initially by
It seems like a good question, but unfortunately I have no familiarity with superforecasting, having never learned about it or participated in anything related except by reading some superficial descriptions of what it is.
Until Feb 2020 I had little interest in making empirical forecasts, since I didn't see it as part of my intellectual interests, and believed in EMH or didn't think it would be worth my time/effort to try to beat the market, so I just left such forecasting to others and deferred to other people who seem to have good epistemics.
If I had to guess based on my shallow understanding of superforecasting, I would say while there are probably overlapping skills, there's a strategic component to trading, which involves things like which sectors to allocate attention to, how to spot the best opportunities and allocate capital to them, while not taking too much concentrated risk, explore vs exploit type decisions, which are not part of superforecasting.