As we approach AGI, we also approach the risk of alignment failure: due to either mismatch between intended goals and specified goals ('outer misalignment'), or mismatch between specified goals and emergent goals ('inner misalignment'), we end up with a catastrophically powerful failure mode that puts us all in a very bad place.
Right now, we don't know what kind of model will lead to AGI. We can guess, but a few years back people didn't have a lot of hope for LLMs, and now look where we're at. A lot of people were deeply surprised. Solutions to alignment failure may be model dependent, and AGI may emerge from something entirely new, or something... (read 286 more words →)
I personally don't find anything on the list disagreeable (including the summarization and mentoring items).
Summarization is a pretty well established memory consolidation technique to improve long-term recall of information. The OP does not explicitly state this is the aim, but that was my assumption, and if so I think it is uncontroversial that that is beneficial.
Regarding the mentoring, the item on the list was "you would have a good mentor" (which I agree with) and then underneath is "One way to do this is to email people" (which I also agree with in the sense that emailing is clearly one way to do this - I do not necessarily feel this is... (read more)