Cole Wyeth — LessWrong

I am a PhD student in computer science at the University of Waterloo, supervised by Professor Ming Li and advised by Professor Marcus Hutter.

My current research is related to applications of algorithmic probability to sequential decision theory (universal artificial intelligence). Recently I have been trying to start a dialogue between the computational cognitive science and UAI communities. Sometimes I build robots, professionally or otherwise. Another hobby (and a personal favorite of my posts here) is the Sherlockian abduction master list, which is a crowdsourced project seeking to make "Sherlock Holmes" style inference feasible by compiling observational cues. Give it a read and see if you can contribute!

See my personal website colewyeth.com for an overview of my interests and work.

I do ~two types of writing, academic publications and (lesswrong) posts. With the former I try to be careful enough that I can stand by ~all (strong/central) claims in 10 years, usually by presenting a combination of theorems with rigorous proofs and only more conservative intuitive speculation. With the later, I try to learn enough by writing that I have changed my mind by the time I'm finished - and though I usually include an "epistemic status" to suggest my (final) degree of confidence before posting, the ensuing discussion often changes my mind again. As of mid-2025, I think that the chances of AGI in the next few years are high enough (though still <50%) that it’s best to focus on disseminating safety relevant research as rapidly as possible, so I’m focusing less on long-term goals like academic success and the associated incentives. That means most of my work will appear online in an unpolished form long before it is published.

Semantics; it’s obviously not equivalent to physical violence.

I expect this to start not happening right away.

So at least we’ll see who’s right soon.

I think if someone is very well-known their making a particular statement can be informative in itself, which is probably part of the reason it is upvoted.

As one of many mitigations it’s better than nothing but it is not sufficient because it does not solve the underlying problems and can easily be subverted by a superintelligence.

Those problems don’t sound new and also don’t seem that relevant to navigating AGI.

My girlfriend’s boss found it convincing, starting from no set opinion.

EA is not well-described as thinking wisdom is compassion. That could be said of many charitable movements, and is less true of EA than typically.

Klein comes off very sensibly. I don’t agree with his reasons for hope, but they do seem pretty well thought out and Yudkowsky did not answer them clearly.

The graph is slightly more informative: https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/

I think it's fair to say I predicted this - I expected exponential growth in task length to become a sigmoid in the short term:

In particular, I expected that Claude's decreased performance on Pokemon with Sonnet 4.5 indicated that it's task length would not be very high. Certainly not 30 hours, though I understand that Anthropic did not claim 30 hours human equivalent, I still believe that their claim of 30 hours continuous software engineering seems dubious - what exactly does that number actually mean, if it does not indicate even 2 hours of human-equivalent autonomy? I can write a program that "remains coherent" while "working continuously" for 30 hours by simply throttling GPT-N to work very slowly by throttling tokens/hour for any N >= 3. This result decreases my trust in Anthropic's PR machine (which was already pretty low).

To be clear, this is only one data point and I may well be proven wrong very soon.

However, I think we can say that the “faster exponential” for inference scaling which some people expected did not hold up.

That’s interesting - I used to play with my brother’s cube but never thought of building a cube as part of the game.

LESSWRONG
LW

LESSWRONG
LW

Sequences

Posts

Wikitag Contributions

Comments