This is a special post for quick takes by leogao. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
what's the strongest case for why something IDA-like cannot be aligned? my current best story for how ambitious mechinterp might be possible depends on an IDA-like assumption. it would be good to know now if this is a doomed endeavor.
I don't think it should be possible to prove that it cannot be aligned, but my default position is that there aren't very strong reasons to believe that the thing will be aligned, the straightforward intuition being that either/both of the distillation or the amplification changes your alignment target and iterating this process amplifies the badness of the drift.
From this it follows that if you actually have some method of either (making sure that things are still pointed at the alignment target) or (making sure that distillation and amplification don't change the alignment target), then this could work, but these subproblems aren't obviously easier than the alignment problem writ large.