LESSWRONG
LW

Tamay — LessWrong

I had in mind just your original counterparty. Requiring become a public market maker seems like quite the commitment.

Tamay from Epoch AI here.

We made a mistake in not being more transparent about OpenAI's involvement. We were restricted from disclosing the partnership until around the time o3 launched, and in hindsight we should have negotiated harder for the ability to be transparent to the benchmark contributors as soon as possible. Our contract specifically prevented us from disclosing information about the funding source and the fact that OpenAI has data access to much but not all of the dataset. We own this error and are committed to doing better in the future.

For future collaborations, we will strive to improve transparency wherever possible, ensuring contributors have clearer information about funding sources, data access,... (read more)

128

-6

•••

Tamay1y*Quick Take

Short version: The claim that AI automation of software engineering will erase NVIDIA's software advantage misunderstands that as markets expand, the rewards for further software improvements grow substantially. While AI may lower the cost of matching existing software capabilities, overall software project costs are likely to keep increasing as returns on optimization rise. Matching the frontier of performance in the future will still be expensive and technically challenging, and access to AI does not necessarily equalize production costs or eliminate NVIDIA's moat.

I often see the argument that, since NVIDIA is largely software, when AI automates software, NVIDIA will have no moat, and therefore NVIDIA a bad AI bet. The argument goes something... (read 376 more words →)

-3

FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI

Tamay

FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI

FrontierMath presents hundreds of unpublished, expert-level mathematics problems that specialists spend days solving. It offers an ongoing measure of AI complex mathematical reasoning progress.

We’re introducing FrontierMath, a benchmark of hundreds of original, expert-crafted mathematics problems designed to evaluate advanced reasoning capabilities in AI systems. These problems span major branches of modern mathematics—from computational number theory to abstract algebraic geometry—and typically require hours or days for expert mathematicians to solve.

To understand and measure progress in artificial intelligence, we need carefully designed benchmarks that can assess how well AI systems engage in complex scientific reasoning. Mathematics offers a unique opportunity for this assessment—it requires extended... (read 795 more words →)

Replying toAlgorithmic Improvement Is Probably Faster Than Scaling Now

Tamay2y*

Algorithmic Improvement Is Probably Faster Than Scaling Now

My guess is that compute scaling is probably more important when looking just at pre-training and upstream performance. When looking innovations both pre- and post-training and measures of downstream performance, the relative contributions are probably roughly evenly matched.

Compute for training runs is increasing at a rate of around 4-5x/year, which amounts to a doubling every 5-6 months, rather than every 10 months. This is what we found in the 2022 paper, and something we recently confirmed using 3x more data up to today.

Algorithms and training techniques for language models seem to improve at a rate that amounts to a doubling of 'effective compute' about doubling every 8 months, though, like our work... (read more)

Replying toAI #60: Oh the Humanity

Tamay2y

AI #60: Oh the Humanity

Sebastian Borgeaud, one of the lead authors of the Chinchilla scaling paper, admits there was a bug in their code. https://twitter.com/borgeaud_s/status/1780988694163321250

Claim that the Chinchilla paper calculated the implied scaling laws incorrectly. Yes, it seems entirely plausible that there was a mistake, tons of huge training runs relied on the incorrect result, and only now did someone realize this. Why do you ask?

Replying toTransformative AGI by 2043 is <1% likely

Tamay2y

Transformative AGI by 2043 is <1% likely

I'm interested. What bets would you offer?

Announcing Epoch's newly expanded Parameters, Compute and Data Trends in Machine Learning database

Robi Rahman

Robi Rahman, Jaime Sevilla Molina, Tamay, Ege Erdil, Pablo Villalobos, Ben Cottier, Matthew Barnett

The performance of machine learning models is closely related to their amount of training data, compute, and number of parameters. At Epoch, we’re investigating the key inputs that enable today’s AIs to reach new heights.

Our recently expanded Parameter, Compute and Data Trends database traces these details for hundreds of landmark ML systems and research papers.

In the past six months, we’ve added 240 new language models and 170 compute estimates. We will be maintaining this dataset, updating it with more historical information, and adding new significant releases. It's a valuable resource for journalists, academics, policymakers, and anyone interested in understanding the trajectory of AI.

Explore the interactive visualization, check out the documentation, and access the data for your own research at epochai.org/data/pcd.

Tamay3y*Quick Take

There is an insightful literature that documents and tries to explain why large incumbent tech firms fail to invest appropriately in disruptive technologies, even when they played an important role in its invention. I speculatively think this sheds some light on why we see new firms such as OpenAI rather than incumbents such as Google and Meta leading the deployment of recent innovations in AI, notably LLMs.

Disruptive technologies—technologies that initially fail to satisfy existing demands but later surpass the dominant technology—are often underinvested in by incumbents, even when these incumbents played a major role in their invention. Henderson and Clark, 1990 discuss examples of this phenomenon, such as Xerox's failure to exploit... (read 519 more words →)

Predicting GPU performance

Marius Hobbhahn

Marius Hobbhahn, Tamay

We develop a simple model that predicts progress in the performance of field-effect transistor-based GPUs under the assumption that transistors can no longer miniaturize after scaling down to roughly the size of a single silicon atom. We construct a composite model from a performance model (a model of how GPU performance relates to the features of that GPU), and a feature model (a model of how GPU features change over time given the constraints imposed by the physical limits of miniaturization), each of which are fit on a dataset of 1948 GPUs released between 2006 and 2021. We find that almost all progress can be explained by two variables: transistor size and... (read 175 more words →)

Revisiting algorithmic progress

Tamay

Tamay, Ege Erdil

How much progress in ML depends on algorithmic progress, scaling compute, or scaling relevant datasets is relatively poorly understood. In our paper, we make progress on this question by investigating algorithmic progress in image classification on ImageNet, perhaps the most well-known test bed for computer vision.

Using a dataset of a hundred computer vision models, we estimate a model—informed by neural scaling laws—that enables us to analyse the rate and nature of algorithmic advances. We use Shapley values to produce decompositions of the various drivers of progress computer vision and estimate the relative importance of algorithms, compute, and data.

Our main results include:

Every nine months, the introduction of better algorithms contributes the equivalent of

... (read 342 more words →)

Replying toWill we run out of ML data? Evidence from projecting dataset size trends

Tamay3y*

Will we run out of ML data? Evidence from projecting dataset size trends

If the data is low-quality and easily distinguishable from human-generated text, it should be simple to train a classifier to spot LM-generated text and exclude this from the training set. If it's not possible to distinguish, then it should be high-enough quality so that including it is not a problem.

ETA: As people point out below, this comment was glib and glosses over some key details; I don't endorse this take anymore.

Replying toThe longest training run

Tamay3y*

The longest training run

Good question. Some thoughts on why do this:

Our results suggest we won't be caught off-guard by highly capable models that were trained for years in secret, which seems strategically relevant for those concerned with risks
We looked whether there was any 'alpha' in these results by investigating the training durations of ML training runs, and found that models are typically trained for durations that aren't far off from what our analysis suggests might be optimal (see a snapshot of the data here)
It independently seems highly likely that large training runs would already be optimized in this dimension, which further suggests that this has little to no action-relevance for advancing the frontier

The longest training run

Jsevillamol

Jsevillamol, Tamay, Owen D, anson.ho

In short: Training runs of large Machine Learning systems are likely to last less than 14-15 months. This is because longer runs will be outcompeted by runs that start later and therefore use better hardware and better algorithms. [Edited 2022/09/22 to fix an error in the hardware improvements + rising investments calculation]

Scenario	Longest training run
Hardware improvements	3.55 years
Hardware improvements + Software improvements	1.22 years
Hardware improvements + Rising investments	9.12 months
Hardware improvements + Rising investments + Software improvements	2.52 months

Larger compute budgets and a better understanding of how to effectively use compute (through, for example, using scaling laws) are two major driving forces of progress in recent Machine Learning.

There are many ways to increase your effective compute budget:... (read 2657 more words →)

Trends in GPU price-performance

Marius Hobbhahn

Marius Hobbhahn, Tamay

Executive Summary

Using a dataset of 470 models of graphics processing units (GPUs) released between 2006 and 2021, we find that the amount of floating-point operations/second per $ (hereafter FLOP/s per $) doubles every ~2.5 years. For top GPUs, we find a slower rate of improvement (FLOP/s per $ doubles every 2.95 years), while for models of GPU typically used in ML research, we find a faster rate of improvement (FLOP/s per $ doubles every 2.07 years). GPU price-performance improvements have generally been slightly slower than the 2-year doubling time associated with Moore’s law, much slower than what is implied by Huang’s law, yet considerably faster than was generally found in prior work on... (read 251 more words →)

Announcing Epoch: A research organization investigating the road to Transformative AI

Jsevillamol

Jsevillamol, Pablo Villalobos, Tamay, lennart, Marius Hobbhahn, anson.ho

Summary

We are a new research organization working on investigating trends in Machine Learning and forecasting the development of Transformative Artificial Intelligence
This work is done in close collaboration with other organizations, like Rethink Priorities, Open Philanthropy, and MIT CSAIL
We will be hiring for 2-4 full-time roles this summer – more information here
You can find up-to-date information about Epoch on our website

What is Epoch?

Epoch is a new research organization that works to support AI strategy and improve forecasts around the development of Transformative Artificial Intelligence (TAI) – AI systems that have the potential to have an effect on society as large as that of the industrial revolution.

Our founding team consists of seven members –... (read 550 more words →)

Replying toIs AI Progress Impossible To Predict?

Tamay4y

Is AI Progress Impossible To Predict?

I'm not sure what you mean; I'm not looking at log-odds. Maybe the correlation is an artefact from noise being amplified in log-space (I'm not sure), but it's not obvious to me that this isn't the correct way to analyse the data.

A concrete bet offer to those with short AGI timelines

Matthew Barnett

Matthew Barnett, Tamay

[Update 4 (12/23/2023): Tamay has now conceded.]

[Update 3 (3/16/2023): Matthew has now conceded.]

[Update 2 (11/4/2022): Matthew Barnett now thinks he will probably lose this bet. You can read a post about how he's updated his views here.]

[Update 1: we have taken this bet with two people, as detailed in a comment below.]

Recently, a post claimed,

it seems very possible (>30%) that we are now in the crunch-time section of a short-timelines world, and that we have 3-7 years until Moore's law and organizational prioritization put these systems at extremely dangerous levels of capability.

We (Tamay Besiroglu and I) think this claim is strongly overstated, and disagree with the suggestion that “It's time for EA... (read 1234 more words →)

120

199

Here’s a rough description of an idea for a betting procedure that enables people who disagree about long-term questions to make bets, despite not wanting to commit to waiting until the long-term questions are resolved.

Suppose person A and person B disagree about whether P, but can’t find any clear concrete disagreements related to this question that can be decided soon. Since they want to bet on things that pay out soon (for concreteness say they only want to bet on things that can pay out within 5 years), they don’t end up betting on anything.

What they can do is they could agree to bet on P, and enter into a contract (or... (read more)

Tamay's Shortform

Tamay

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

Projecting compute trends in Machine Learning

Tamay

Tamay, lennart, Jsevillamol

Summary

Using our dataset of milestone Machine Learning models, and our recent analysis of compute trends in ML, we project forward 70 years worth of trends in the amount of compute used to train Machine Learning models. Our simulations account for (a) uncertainty in estimates of the growth rates in compute usage during the Deep Learning (DL)-era and Pre-DL era, and (b) uncertainty over the ‘reversion date’, i.e. the date when the current DL-era compute trend (with a ~6 month doubling time) will end and revert to the historically more common trend associated with Moore’s law. Assuming a reversion date of between 8 to 18 years, and without accounting for algorithmic progress, our... (read 1582 more words →)