This is a special post for quick takes by Cole Wyeth. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I think that maybe continual learning works fine and the models are just myopic?
myopia can easily be confused with broken continual learning because both would have a common(ly measured) effect: limiting task horizon lengths.
Also, myopic models would tend to underinvest in cleaning up their contexts to remain stable in the long term, which might look like a continual learning failure.
An alternative (and hard-to-distinguish) explanation is that models lack some kind of executive functioning ability. There can probably be a lot of overlap between these things.
By “continual learning works fine” I mean sufficient for AGI and probably ASI. I think AI continual learning currently lags humans, but maybe just because the models are lazy about learning stuff, and smarter/less myopic models with the same continual learning implementations used today + some light gradient updating can probably get there. I don’t mean that there’s no better continual learning algorithm.
I’ve thought about this for awhile but only now guess it’s net positive to share, because many on lesswrong seem to expect a sharp transition to a dangerous regime “once continual learning is solved.“ I’ll take this down if others think it’s net negative though.
How widespread is this belief?
I can imagine a sharp transition if continual learning comes from one brilliant insight. It seems a bit more likely that we'll get a gradual transition due to multiple small kludgy advances.