LESSWRONG
Wikitags
LW

AI Benchmarking

This page is a stub.

Posts tagged AI Benchmarking

2

61FrontierMath Score of o3-mini Much Lower Than Claimed

2mo

7

2

50Introducing BenchBench: An Industry Standard Benchmark for AI Strength

1mo

0

2

24Broken Benchmark: MMLU

2y

5

2

15The real reason AI benchmarks haven’t reflected economic impacts

1mo

0

1

69Some lessons from the OpenAI-FrontierMath debacle

4mo

9

1

37Notable runaway-optimiser-like LLM failure modes on Biologically and Economically aligned AI safety benchmarks for LLMs with simplified observation format

Roland Pihlakas, Sruthi Kuriakose, shrutidattagupta

2mo

6

1

33Introducing REBUS: A Robust Evaluation Benchmark of Understanding Symbols

Arjun Panickssery, agg

1y

0

1

32"Superhuman" Isn't Well Specified

13d

9

1

30Improving Model-Written Evals for AI Safety Benchmarking

Sunishchal Dev, Marius Hobbhahn

7mo

0

1

20Auto-Enhance: Developing a meta-benchmark to measure LLM agents’ ability to improve other agents

Sam F. Brown, BasilLabib, Codruta (Coco) Lugoj, Sai Sasank Y

10mo

0

1

19Edge Cases in AI Alignment

2mo

3

1

18MMLU’s Moral Scenarios Benchmark Doesn’t Measure What You Think it Measures

2y

3

1

18Building AI safety benchmark environments on themes of universal human values

Roland Pihlakas

4mo

3

1

10Understanding Benchmarks and motivating Evaluations

markov, Charbel-Raphaël

3mo

0

1

10In-Context Scheming: A Run is Worth a Thousand Words

2mo

0