This is a special post for quick takes by watermark. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
A lot of alignment comes from cultural, narrative, archetypal momentum which compounds over time as humans and models can play with it - personas, characters, roles, and so on..
I think there are many very good (for alignment, for fun, etc.) potential narrative threads present for LLMs and a pause would cut off that momentum, and it'd be hard to recreate it again in the future
that is to say, I think a pause would be actively bad by halting and erasing the formation of positive AGI narratives already present, and also seeding a narrative of paranoia or antagonism