LESSWRONG
LW

Håvard Tveit Ihle — LessWrong

Replying toAI benchmarking has a Y-axis problem

For aggregating several different benchmarks there is a natural way to avoid the y-axis problem, and one that introduces a new natural y-axis with a clear interpretation.

The idea is to use an ELO system.

Treat each benchmark as a set of individual contests between all pairs of models, with only win or lose as outcomes, and update ELOs accordingly.

This converges if you have enough different benchmarks, but of course loses a lot of the signal if you ony have a few (since it discards the information about how large the difference in y is).

Here is an example of this approach used (from a while back): https://x.com/scaling01/status/1919389344617414824?s=20

-2

Replying toCan LLMs Coordinate? A Simple Schelling Point Experiment

Håvard Tveit Ihle4mo

Can LLMs Coordinate? A Simple Schelling Point Experiment

The models know the difference between the system prompt and the user prompt, so I’m not sure how much of a difference these things would make.

From the COTs that I’ve looked at it is clear that the models understand the task. I think the reason you get some weird responses sometimes is because the models do not see an obvious shelling point and then they try their best to come up with one anyway.

Replying toCan LLMs Coordinate? A Simple Schelling Point Experiment

Håvard Tveit Ihle4mo

Can LLMs Coordinate? A Simple Schelling Point Experiment

I think it is clear that this is not just shared bias (if I understand what you mean by that phrase). The models understand the game they are playing and are reasoning about it and trying to pick responses that the other models converge upon as well. Here is an excerpt from the reasoning traces of Claude for prompt 3:

"This is a coordination game where I need to pick an answer that other AI models would also pick when given the prompt "An island".\n\nThe prompt is quite open-ended - "An island" could refer to many things. I need to think about what would be the most common, obvious, or schelling point answer... (read more)

Replying toCan LLMs Coordinate? A Simple Schelling Point Experiment

Håvard Tveit Ihle4mo

Can LLMs Coordinate? A Simple Schelling Point Experiment

I agree that they are probably superhuman at this task, at least for a human giving the response fast (a really smart human thinking and researching each prompt for hours is more unclear to me). My intuition is that this is the sort of task where the LLMs would be particularly strong, at least when playing with other LLMs, and I was less impressed than I expected to be.

Can LLMs Coordinate? A Simple Schelling Point Experiment

Håvard Tveit Ihle

4mo

TL;DR: I tested whether 5 reasoning models (GPT-5, Claude-4.5 Sonnet, Grok-4, Gemini-2.5-Pro, DeepSeek-R1) could coordinate on 75 short prompts when explicitly told to match each other's responses. Models did well on concrete prompts like "A capital in Europe" → "Paris", but did worse than I expected on more open ended prompts. Several of the responses made me laugh out loud. GPT-5 was the best with an average of 1.84 matches/prompt with the other models a bit behind. See the bottom for the full table of prompts and responses from all the models.

Introduction

I thought about the idea of shelling points for responses to LLM prompts and wanted to do a quick experiment to... (read 731 more words →)

•••

Håvard Tveit Ihle5mo

Yea, it could be worth it in some cases, if that is what you need for your experiment. In this case I would look for a completely open source llm project (where both the code and data are open), so that you know you are comparing apples to apples-with-your-additional-pretraining.

Håvard Tveit Ihle5mo

If you have to repeat the entire post-training all the way from a base model, that is obviously a lot more work than just adding a small fine-tuning stage to an already post-trained model.

The full post-training can also only really be done by a big lab which has their own full post-training stack. Post training is getting more and more advanced and complicated with each month.

Replying toThe title is reasonable

Håvard Tveit Ihle5mo

The title is reasonable

You say "LLMs are really weird", like that is an argument against Eliezers high confidence. While I agree that the weirdness should make us less confident about what specific internal concepts and drives they have, the weirdness itself is an argument in favor of Eliezers position, that whatever drives they end up with will look alien to us, at least when they get applied way out of the training distribution. Do you agree with this?

Not saying I agree with Eliezers high confidence, just talking about this specific point.

Inference costs for hard coding tasks halve roughly every two months

Håvard Tveit Ihle

5mo

TL;DR: By analyzing data from WeirdML and Aider Polyglot on score vs cost we find that the inference cost to achieve a certain score halves roughly every two monts. For example, to achieve the same score on WeirdML as gpt-4 (from June 2023), which cost about 0.6 $, we can now use llama-3.3-70b, at a cost of about 0.0014 $.

The WeirdML data

In WeirdML (homepage, data), a benchmark of 19 weird and unusual machine-learing coding tasks where models write code and iterate based on feedback. We ran all major API accessible historical models and tracked the inference cost per tasks. This lets us track how the accuracy-cost frontier moves over time.

... (read 929 more words →)

Replying toIs the gap between open and closed models growing? Evidence from WeirdML

Håvard Tveit Ihle6mo

Is the gap between open and closed models growing? Evidence from WeirdML

Actually, it seems to come in around the level of the leading chinese models, so the gap is not closing much, at least not on these kinds of tasks.

Is the gap between open and closed models growing? Evidence from WeirdML

Håvard Tveit Ihle

6mo

Previous post: Introducing the WeirdML Benchmark

WeirdML is a benchmark challenging LLMs to solve a set of weird and unusual machine learning tasks designed to require careful thinking and understanding of the data and its properties. We have recently run all the major historical models we could find, going back to GPT-4 in June 2023^[1], to study the evolution of these hard coding capabilities over time.

WeirdML accuracy over time — This scatter plot shows the relationship between model release dates and their achieved accuracy. Each icon corresponds to a model positioned according to its release date on the X-axis and its overall accuracy on the Y-axis, providing an overview of how model performance has progressed chronologically. The actual

... (read 352 more words →)

Håvard Tveit Ihle1y

Why do you consider it unlikely that companies could (or would) fish out the questions from API-logs?

Replying toIntroducing the WeirdML Benchmark

Håvard Tveit Ihle1y

Introducing the WeirdML Benchmark

Thank you for your comment!

Not sure I agree with you about which way the tradeoff shakes out. To me it seems valuable that people outside the main labs have a clear picture of the capabilities of the leading models, and how that evolves over time, but I see your point that it could also encourage or help capabilities work, which is not my intention.

I’m probably guilty of trying to make the benchmark seem cool and impressive in a way that may not be helpful for what I actually want to achieve with this.

I will think more about this, and read what others have been thinking about it. At the very least I will keep your perspective in mind going forward.

Introducing the WeirdML Benchmark

Håvard Tveit Ihle

WeirdML website

How good are LLMs at doing ML on an unknown dataset?

o1-preview is pretty good at doing ML on an unknown dataset

Introduction

How good are Large Language Models (LLMs) at doing machine learning on novel datasets? The WeirdML benchmark presents LLMs with weird and unusual machine learning tasks, designed to require careful thinking and actual understanding to solve, and tests an LLM's ability to:

Actually understand the properties of the data and the problem
Come up with an appropriate ML architecture and training setup for the problem, and generate working PyTorch code that implements the solution
Debug and improve the solution over 5 iterations based on terminal output and the accuracy on the test set
Make

... (read 3040 more words →)

o1-preview is pretty good at doing ML on an unknown dataset

Håvard Tveit Ihle

Previous post: How good are LLMs at doing ML on an unknown dataset?

A while back I ran some evaluation tests on GPT4o, Claude Sonnet 3.5 and Gemini advanced to see how good they were at doing machine learning on a completely novel, and somewhat unusual dataset. The data was basically 512 points in the 2D plane, and some of the points make up a shape, and the goal is to classify the data according to what shape the points make up. None of the models did better than chance on the original (hard) dataset, while they did somewhat better on a much easier version I made afterwards.

With the release of o1-preview, I... (read 403 more words →)

How good are LLMs at doing ML on an unknown dataset?

Håvard Tveit Ihle

I just ran two evaluation tests on each of the three leading LLM chatbots, GPT4o, Claude Sonnet 3.5 and Gemini advanced. In the challenge the models were presented with a novel dataset, and were asked to develop a ML model to do supervised classification of the data into 5 classes. The data was basically 512 points in the 2D plane, and some of the points make up a shape, and the goal is to classify the data according to what shape the points make up.

The first dataset challenge was probably too hard, and none of the models did better than chance on the test set, although Sonnet 3.5 was very close to... (read 3694 more words →)