I thought a lot about what kinds of things make sense for me to do to solve AI alignment. That did not make me confident that any particular narrow idea that I have will eventually lead to something important.
Rather, I'm confident that executing my research process will over time lead to something good. The research process is:
Take some vague intuitions
Iteratively unroll them into something concrete
Update my models based on new observations I make during this overall process.
I think being confident, i.e. not feeling hopeless in doing anything, is important. The important takeaway here is that you don't need to be confident in any particular idea that you come up with. Instead, you can be confident in the broader picture of what you are doing, i.e. your processes.
The best way to become confident in this way is to just work a bunch and then reflect back. It is very likely that you will be able to see how improved. And probably you will have had a few successes.
This exact thought, from my diary in ~June 2022: "I advocate keeping a clear separation between how confident you are that your plan will work with how confident you are that pursuing the plan is optimal."
[...]
1Johannes C. Mayer
I think this is a useful model. If I understand correctly what you're saying, then it is that for any particular thing we can think about whether that thing is optimal to do, and whether I could get this thing to work seperately.
I think what I was saying is different. I was advocating confidence not at the object level of some concrete things you might do. Rather I think being confident in the overall process that you engage in to make process is a thing that you can have confidence in.
Imagine there is a really good researcher, but now this person forgets everything that they ever researched, except for their methodology. It some sense they still know how to do research. If they fill in some basic factual knowledge in their brain, which I expect wouldn't take that long, I expect they would be able to continue being an effective researcher.
I wrote the entry in the context of the question "how can I gain the effectiveness-benefits of confidence and extreme ambition, without distorting my world-model/expectations?"
I had recently been discovering abstract arguments that seemed to strongly suggest it would be most altruistic/effective for me to pursue extremely ambitious projects; both because 1) the low-likelihood high-payoff quadrant had highest expected utility, but also because 2) the likelihood of success for extremely ambitious projects seemed higher than I thought. (Plus some other reasons.) I figured that I needn't feel confident about success in order to feel confident about the approach.