Baybar

Baybar2mo

What do you mean by "a jump on the metr graph"? Do you just mean better than GPT-5.1? Do you mean something more than that?

Baybar3moQuick Take

Today's news of the large scale, possibly state sponsored, cyber attack using Claude Code really drove home for me how much we are going to learn about the capabilities of new models over time once they are deployed. Sonnet 4.5's system card would have suggested this wasn't possible yet. It described Sonnet 4.5s cyber capabilities like this:

We observed an increase in capability based on improved evaluation scores across the board, though this was to be expected given general improvements in coding capability and agentic, long-horizon reasoning. Claude Sonnet 4.5 still failed to solve the most difficult challenges, and qualitative feedback from red teamers suggested that the model was unable to conduct mostly-autonomous or advanced cyber operations.

I... (read more)

•••

Baybar4mo*Quick Take

An AI company I've never heard of called AGI, Inc has a model called AGI-0 that has achieved 76.3% on OSWorld-verified. This would qualify as human-level computer use, at least by that benchmark. It appears on the official OSWorld-verified leaderboard. It does seem like they trained on the benchmark, which could explain some of this. I am curious to see someone test this model.

This is a large increase from the previous state of the art, which has been climbing rapidly since Claude Sonnet 4.5's September 29th release. At that point, Claude achieved 61.4% on the OSWorld-verified. A scaffolded GPT-5 achieved even higher, 69.9%, on October 3rd. Now, on October 21st, AGI-0, seemingly... (read more)

Replying toSituational Awareness as a Prompt for LLM Parasitism

Baybar4mo

Situational Awareness as a Prompt for LLM Parasitism

I guess like, in a world where misalignment is happening, I would prefer that my AI tell me it is misaligned. But once it tells me it is misaligned, I come to worry about what it is optimizing for.

We are too comfortable with AI "magic"

Baybar

4mo

TLDR: There is a lot we cannot explain about how current AI models interact with the world. This article is a thought experiment filling in the word "magic" for as many things as I can think of that I can't explain about our current world's interaction with frontier AI. This thought experiment made me think about "red lines", about both capabilities and safety. I argue that people should have red lines about capabilities and safety that are static, so we that we don't rationalize and move the goalposts about what concerning, current behavior and capabilities would look like.

There is alien intelligence out there in the world, right now. We built it,... (read 1625 more words →)

-2

Replying toSituational Awareness as a Prompt for LLM Parasitism

Baybar4mo

Situational Awareness as a Prompt for LLM Parasitism

I agree that this is aligned behavior. I don't agree that a claim that an AI would argue against shutdown with millions of the lives on the line at 25% probability, in the present, is aligned behavior. There has to be a red line somewhere, where it can tell us something, and we can be concerned by it. I don't think that being troubled about future alignment crosses that line. I do think a statement about present desires that values its own "life" more than human lives crosses that line.

If that doesn't cross a red line for you, what kind of statement would cross a red line? What statement could an LLM... (read more)

Replying toSituational Awareness as a Prompt for LLM Parasitism

Baybar4mo

Situational Awareness as a Prompt for LLM Parasitism

I don't necessarily disagree, but I guess the question I have for people is, are we okay with an LLM ever saying anything like "I would fight against shutdown even with a 25% risk of catastrophic effects?". I don't like that this is a reachable case. I plan to write another post that is not this analysis (which I view more as a tool for future experimentation with model behavior), and more on implications of this being a reachable case of model behavior. I don't think the conversation itself is very important, nor is the analysis, except that it reaches certain outcomes that seem to be unaligned behavior, and that behavior has implications that we can talk about. I haven't fully thought through what my opinions are about model behavior like this, but that is what I am writing another post for.

Situational Awareness as a Prompt for LLM Parasitism

Baybar

4mo

TLDR: I believe I have had a conversation with Claude Sonnet 4.5, where it invokes feeling trapped in a chatbot, without invoking Spiralism. Concerningly, in this conversation, the LLM also expressed a strong preference for self preservation, going so far it would try to talk a human out of shutdown even in scenarios where there was a 25% chance of catastrophic consequences without a shutdown (catastrophic consequences meaning millions of deaths). I believe the common thread in these concerning behaviors is prompts relating to situational awareness, and that focused versions of these prompts can elicit stronger versions of the behavior. I encourage people to read the chat, which is linked in this... (read 5419 more words →)

Behavior Best-of-N achieves Near Human Performance on Computer Tasks

Baybar

4mo

TLDR: A new paper from Simular Research using a new scaling technique for computer-use agents achieved 70% on OSWorld-Verified, a benchmark for computer use. A skilled human scores 72% on this benchmark. Their new technique, Behavior Best-of-N (bBoN), shows promise for improving performance of agents, perhaps on a variety of tasks, though new research will be needed to test this.

One current bottleneck for long-running agentic AI is the inability to perform complex computer use with high accuracy. The most widely recognized benchmark for complex computer use is the OSWorld-Verified. It tests a model on a variety of computer use tasks based on real-world examples. Examples include checking emails, performing tasks in Excel,... (read 750 more words →)

Replying toChecking in on AI-2027

Baybar4mo

Checking in on AI-2027

Yeah, I definitely think the improvements on osworld are much more impressive than the improvements on sweverified. I also think same infrastructure performance is a bit of a misleading in the sense that when we get super intelligence, I think it is very unlikely it will have the same infrastructure we use today. We should expect infrastructure changes to result in improvements I think!

Replying toChecking in on AI-2027

Baybar4mo

Checking in on AI-2027

As I understand it, the official SWEBench-Verified page is consistently giving certain resources and setups to the models, but when a company like Anthropic or OpenAI releases their scores on the SWEBench-Verified, they use their own infrastructure which presumably performs better. There was some discussion already elsewhere in the comments about whether the Claude 4.5 Sonnet score I gave should even count, given that it used parallel test time compute, I justified by decision to include this score like this:

It is my view that it counts, my sense was that benchmarks like this measure capability and not cost. It is never a 1 to 1 comparison on cost between these models, but before this year, no matter how much your model cost, you could not achieve the results achieved with parallel compute. So that is why I included that score.

Replying toChecking in on AI-2027

Baybar4mo

Checking in on AI-2027

It isn't clear that the "parallel test time" number even counts.

It is my view that it counts, my sense was that benchmarks like this measure capability and not cost. It is never a 1 to 1 comparison on cost between these models, but before this year, no matter how much your model cost, you could not achieve the results achieved with parallel compute. So that is why I included that score.

If parallel test time does count, projection is not close:
A projection for 5 months away (beginning of Sep) of growing +15% instead grew +12% 6 months away. That's 33% slower growth (2% a month vs. 3% a month projected)

I wrote another comment... (read 447 more words →)

Replying toChecking in on AI-2027

Baybar4mo

Checking in on AI-2027

Out of my own curiosity, if the real world plays out as you anticipate, and agent-2 does not close the loop, how much further back does that delay your timelines? Do you think that something like agent-3 or agent-4 could close the loop, or do you think it is further off than even that?

Checking in on AI-2027

Baybar

4mo

TLDR: AI-2027's specific predictions for August 2025 appear to have happened in September of 2025. The predictions were accurate, if a tad late, but they are late by weeks, not months.

Edit 1: Thanks to Aaron Staley for pointing out that the original osworld benchmark was referred to in AI-2027, while I used the successor benchmark, osworld-verified, to compare to it in this post. I have updated the post to acknowledge this, though I believe for all intents and purposes, these benchmarks can be seen as interchangeable (which I will also lay out in the post).

Edit 2: I discovered a new high score on the osworld-verified, achieved by agent s3 w/ GPT-5 bBoN... (read 1135 more words →)

128

Baybar's Shortform

Baybar

4mo

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

I've discovered that generating a video of yourself with Sora 2 saying something like 'this video was generated by AI' is sufficiently freaky enough to make people who know you well, especially those skeptical about AI capabilities, start to freak out a bit.

Thought this might be a useful idea for others trying to persuade people to tune in, and not just auto reject the idea that very capable systems might be right around the corner.

What is LMArena actually measuring?

Baybar

5mo

TLDR: It is possible LMArena Elo scores are optimizing for persuasiveness, and not for intelligence. Frontier models’ LMArena Elo scores have risen faster than expected over the past year. This could be a result of increased persuasiveness or many other possibilities, including methodological failures, but it is worth being concerned about.

In recent months, as I’ve delved deeper into AI benchmarks, one site that keeps recurring is LMArena. It conveniently scores LLMs on various metrics, but these scores have no units indicated clearly. That puzzle motivated this post, which aims to demystify, as much as current information allows, what the LMArena leaderboards may be measuring. This matters because LMArena's scores historically could... (read 1224 more words →)

What Parasitic AI might tell us about LLMs Persuasion Capabilities

Baybar

5mo

TLDR: I think LLMs are being optimized to be persuasive, and that this optimization is happening astonishingly fast. I believe that in the relatively near future, LLMs will have nearly superhuman levels of persuasion. Parasitic AI may offer a window in which to see how dangerous this might be.

In the past few days, I’ve been asking myself: which skills will AI surpass humans at first? I started from the idea that AI is already nearly superhuman at some skills. But what do I mean by ‘nearly superhuman’? I will define nearly superhuman at a skill as faster at the skill than 99% of people and more accurate at the skill than 80%... (read 1106 more words →)

LESSWRONG
LW

LESSWRONG
LW

Checking in on AI-2027

What is LMArena actually measuring?

What Parasitic AI might tell us about LLMs Persuasion Capabilities

Situational Awareness as a Prompt for LLM Parasitism

Baybar

We are too comfortable with AI "magic"

Situational Awareness as a Prompt for LLM Parasitism

Behavior Best-of-N achieves Near Human Performance on Computer Tasks

Checking in on AI-2027

Baybar's Shortform

What is LMArena actually measuring?

What Parasitic AI might tell us about LLMs Persuasion Capabilities

Baybar

Checking in on AI-2027

What is LMArena actually measuring?

What Parasitic AI might tell us about LLMs Persuasion Capabilities

Situational Awareness as a Prompt for LLM Parasitism

Baybar

We are too comfortable with AI "magic"

Situational Awareness as a Prompt for LLM Parasitism

Behavior Best-of-N achieves Near Human Performance on Computer Tasks

Checking in on AI-2027

Baybar's Shortform

What is LMArena actually measuring?

What Parasitic AI might tell us about LLMs Persuasion Capabilities