I don't agree with the abrupt takeoff assumption. I think a lot of the evidence people use to support that assumption falls apart after further scrutiny. Some examples:
Thanks! This is very helpful, and yes, I did mean to refer to grokking! Will update the post.
Thanks! Am probably convinced by the third point, unsure about the others due to not having much time to think at the moment.
I think it would be a good idea to ask the question at the ongoing thread on AGI safety questions.
(note: Quickly written. I've attempted to number my arguments "formally", but I have no training in this format. Edits/suggestions welcome.)
thus
Argument for claim 1:
thus
Evidence for claim 4 — "Narrow" lumpy generality — specifically, new and powerful abstraction gives rise to abrupt domain-specific performance improvements:
Evidence for claim 5 — cognitive architecture components generalize:
Evidence for claim 2 — takeoff is threshold-based: