How scarce are tickets/"seats"?

I will carefully hedge my investment in this company by giving it $325823e7589245728439572380945237894273489, in exchange for a board seat so I can keep an eye on it.

I have over 5 Twitter followers, I'll take my board seat when ur ready

Giving up on transhumanism as a useful idea of what-to-aim-for or identify as, separate from how much you personally can contribute to it.

More directly: avoiding "pinning your hopes on AI" (which, depending on how I'm supposed to interpret this, could mean "avoiding solutions that ever lead to aligned AI occurring" or "avoiding near-term AI, period" or "believing that something other than AI is likely to be the most important near-future thing", which are pretty different from each other, even if the end prescription for you personally is (or seems, on first pass, to be) the same.), separate from how much you personally can do to positively affect AI development.

Then again, I might've misread/misinterpreted what you wrote. (I'm unlikely to reply to further object-level explanation of this, sorry. I mainly wanted to point out the pattern. It'd be nice if your reasoning did turn out correct, but my point is that its starting-place seems/seemed to be rationalization as per the pattern.)

Yes, I think this post / your story behind it, is likely an example of this pattern.

That's technically a different update from the one I'm making. However, I also update in favor of that, as a propagation of the initial update. (Assuming you mean "good enough" as "good enough at pedagogy".)

This sure does update me towards "Yudkowsky still wasn't good enough at pedagogy to have made 'teach people rationality techniques' an 'adequately-covered thing by the community'".

• Person tries to work on AI alignment.
• Person fails due to various factors.
• Person gives up working on AI alignment. (This is probably a good move, when it's not your fit, as is your case.)
• Danger zone: In ways that sort-of-rationalize-around their existing decision to give up working on AI alignment, the person starts renovating their belief system around what feels helpful to their mental health. (I don't know if people are usually doing this after having already tried standard medical-type treatments, or instead of trying those treatments.)
• Danger zone: Person announces this shift to others, in a way that's maybe and/or implicitly prescriptive (example).

There are, depressingly, many such cases of this pattern. (Related post with more details on this pattern.)

Group Debugging is intriguing...

How many times has someone expressed "I'm worried about 'goal-directed optimizers', but I'm not sure what exactly they are, so I'm going to work on deconfusion."? There's something weird about this sentiment, don't you think?

I disagree, and I will take you up on this!

"Optimization" is a real, meaningful thing to fear, because:

• We don't understand human values, or even necessarily meta-understand them.
• Therefore, we should be highly open to the idea that a goal (or meta-goal) that we encode (or meta-encode) would be bad for anything powerful to base-level care about.
• And most importantly, high optimization power breaks insufficiently-strong security assumptions. That, in itself, is why something like "security mindset" is useful without necessarily thinking of a powerful AI as an "enemy" in war-like terms.
  • Here "security assumptions" is used in a broad sense, the same way that "writing assumptions" (the ones needed to design a word-processor software) could include seemingly-trivial things like "there is an input device we can access" and "we have the right permissions on this OS".