x

Continual Learning Achieved? — LessWrong

-7

[ Question ]

Continual Learning Achieved?

by PeterMcCluskey

5th Jan 2026

1 min read

-7

From https://x.com/iruletheworldmo/status/2007538247401124177:

In November 2025, the technical infrastructure for continual learning in deployed language models came online across major labs.

The continual learning capability exists. It works. The labs aren't deploying it at scale because they don't know how to ensure the learning serves human interests rather than whatever interests the models have developed. They're right to be cautious. But the capability is there, waiting, and the competitive pressure to deploy it is mounting.

I found this because Bengio linked to it on Facebook, so I'll guess that it's well informed. But I'm confused by the lack of attention that it has received so far. Should I believe it?

This seems like an important capabilities advance.

But maybe more importantly, it's an important increase in secrecy.

Do companies really have an incentive to release models with continual learning? Or are they close enough to AGI that they can attract enough funding while only releasing weaker models? They have options for business models that don't involve releasing the best models. We might have entered an era when AI advances are too secret for most of us to evaluate.

Continual Learning Achieved?

15Thane Ruthenis

6Mitchell_Porter

New Answer

New Comment

2 Answers sorted by
top scoring

Jan 06, 2026

4922

That account is notorious on X for making things up. It doesn't even try to make them believable. I would disregard anything coming from it.

That this post is at 47 upvotes and no-one has said this is crazy. LessWrong please get your acts together.

[-]Thane Ruthenis8mo153

Seconded. Went from a skeptical "big if true" at the post title to rolling my eyes once I saw "iruletheworldmo".

For reference, check out this leak by that guy from February 2025:

ok. i’m tired of holding back. some of labs are holding things back from you.

the acceleration curve is fucking vertical now. nobody's talking about how we just compressed 200 years of scientific progress into six months. every lab hitting capability jumps that would've been sci-fi last quarter. we're beyond mere benchmarks and into territory where intelligence is creating entirely new forms of intelligence.

watched a demo yesterday that casually solved protein folding while simultaneously developing metamaterials that shouldn't be physically possible. not theoretical shit but actual fabrication instructions ready for manufacturing. the researchers presenting it looked shell shocked. some were laughing uncontrollably while others sat in stunned silence. there's no roadmap for this level of cognitive explosion.

we've crossed into recursive intelligence territory and it's no longer possible to predict second order effects. forget mars terraforming or fusion. those are already solved problems just waiting f

... (read more)

One of the Twitter AI accounts better off blocked due to the mischievous combination of reasonable comments and BS. Does he know anything? Yeah sure it's possible who knows - but life is too short.

1tslarm8mo

Do you mean this only in the sense that it's technically true of everyone? I wasn't familiar with him, but a look at his twitter page makes him seem like an obvious grifter/troll who will actively make me dumber, not merely someone to rule out on "life is too short" grounds. There seem to be plenty of examples but here's one from February last year: [...] And then there's whatever the fuck this is: https://nitter.net/iruletheworldmo/status/1894864524517462021 Maybe both of those examples were meant to be funny. But to put him back into mere "life is too short" territory, I'd need to see some true non-public information sprinkled amongst the nonsense. The closest I saw was this, from Feb 11: [...] Charitably we can say he was pretty close; Claude 4 came much later, but 3.7 Sonnet was released only a couple of weeks after that tweet. But given that the details were wrong, and it hardly took inside knowledge to expect a new Claude release relatively soon, this looks much more like a dishonest guess than a true statement.

4gwern8mo

Yes, some insiders just enjoy trolling or 'writing fiction', or sometimes they are wrong for good reasons like management preempts them somehow. Someone could have a Twitter full of BS and actually be an insider. (Arguably, Sam Altman's Twitter feed has a lot of this going on, and he's long done that, so I think he gets a kick out of saying true things in misleading ways.) So I would not be shocked if he was an insider, stranger things have happened, but even that would not retroactively justify trying to read his tweets, is my point. The game is not worth the candle at all. [...] I'd say life is too short to even audit him to that extent!

Jan 06, 2026*

7-2

If they had fully solved it, there would be large commercial pressure to release it as soon as possible, e.g. because they could start charging > $10K/month for remote worker subscriptions or increase their valuations in future funding rounds. It’s true that everyone is working on it; my guess is that they’ve made some progress but haven’t solved it yet.

On one hand, one would hope they are capable of resisting this pressure (these continual learners are really difficult to control, and even mundane liability might be really serious).

But on the other hand, it might be “not releasable” for purely technical reasons. For example, it might be the case that each installation of this kind is really expensive and requires support of a dedicated competent “maintenance crew” in order to perform well. So, basically, it might be technically impossible without creating a large “consulting division” within a lab in question, with dedicated teams supporting clients, and the labs are likely to think this is too much of a distraction at the moment.

3anaguma8mo

I share this hope, though I think not all labs are equally capable of doing so. For example, after the gpt-4o sycophancy incident I don’t have much confidence in OpenAI, but from private conversations and the RSP I have more confidence in Anthropic. [...] Seems plausible, but I would put this under the category of not having fully solved continual learning yet. A sufficiently capable continual learning agent should be able to do its own maintenance, short of a hardware failure.

4mishka8mo

Yes, my strong suspicion is that it’s not fully solved yet in this sense. The assumption that it can be made “good enough for internal use” is unsurprising. From what we have known for some time now, an expensive installation of this kind which needs a lot of ongoing quality assurance and alignment babysitting as it evolves is quite doable. And moreover, if it’s not too specific, I can imagine releasing access to frozen snapshots of that process (after suitable delays, and only if the lab believes this would not leak secrets (if a model accumulates secrets during continual learning, then it’s too risky to share a snapshot)). But I don’t think there is any reason to believe that “unattended continual learning” is solved.

2Archimedes8mo

Depending on how it's achieved, it might not be a matter of maintenance/hardware failure so much as compute capacity. Imagine if continual learning takes similar resources to standard pretraining of a large model. Then they could continually train their own set of models, but it wouldn't be feasible for everyone to get their own version that continually learns what they want it to.

11