This is a special post for quick takes by Roman Malov. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Mech interp is just crawling out of the hole that deep learning has put us in.
The classical problems of ASI alignment still remain even in case with perfectly readable, human-written code.
In fact, in the old times of MIRI, they were not expecting NNs to work, but still expected a bunch of problems with constructing safe (or, as they said at that time, Friendly) ASI.
But right now I'm not that sure. Pretrained LLMs can be prompted/finetuned to have some of the "common-sense reasonableness" (and therefore avoid the problems with goodharty misinterpreting of your wish with hidden complexity) which might be hard to hardcode, so advanced interp could ease this part out. What are your thoughts on this "agenda"?