LESSWRONG
LW

charlieoneill — LessWrong

Yes - the general argument is "task length isn't sufficiently correlated with actual use for remote work, so you also need to look at other things" (see the EpochAI post on this)

Replying toFive Hinge‑Questions That Decide Whether AGI Is Five Years Away or Twenty

charlieoneill9mo

Five Hinge‑Questions That Decide Whether AGI Is Five Years Away or Twenty

Thanks for being transparent; wouldn't put too much stock in a short-term rise in Coreweave (as I understand, more to do with tariffs and other confounding factors than "they were underpriced before")

Five Hinge‑Questions That Decide Whether AGI Is Five Years Away or Twenty

charlieoneill

9mo

For people who care about falsifiable stakes rather than vibes

TL;DR

All timeline arguments ultimately turn on five quantitative pivots. Pick optimistic answers to three of them and your median forecast collapses into the 2026–2029 range; pick pessimistic answers to any two and you drift past 2040. The pivots (I think) are:

Which empirical curve matters (hardware spend, algorithmic efficiency, or revenue)
Whether software‑only recursive self‑improvement (RSI) can accelerate capabilities faster than hardware can be installed.
How sharply compute translates into economic value once broad “agentic” reliability is reached.
Whether automating half of essential tasks ignites runaway growth or whether Baumol’s law keeps aggregate productivity anchored until all bottlenecks fall
How much alignment fear, regulation, and supply‑chain friction slow scale‑up

The rest... (read 1339 more words →)

127

•••

Born on Third Base: The Case for Inheriting Nothing and Building Everything

charlieoneill

The idea that wealth should be inherited is so ingrained in our thinking that we rarely question it. But step back for a moment, and it becomes a curious thing—this notion that assets, land, fortunes should be passed down from one generation to the next without scrutiny. The debate over wealth distribution is usually framed in extremes: on one side, Marxism, condemning private inheritance as exploitation, advocating for collective ownership. On the other, laissez-faire capitalism, defending property rights even when they entrench privilege. A 100% death tax emerges as a synthesis of these views rather than an attack on individual success. It preserves incentives for personal achievement while preventing wealth from pooling... (read 498 more words →)

-24

One-dimensional vs multi-dimensional features in interpretability

charlieoneill

Chris Olah's “What is a Linear Representation? What is a Multidimensional Feature?” (July Circuits Update) prompted a moment of pause for me regarding the term "one-dimensional feature." I initially conflated that phrase with the number of dimensions in the activation space (for example, the 768 dimensions in GPT‑2 Small). However, Olah uses "one-dimensional" to describe a property of the feature's structure - not the width of the activation vector. In this post, I clarify this distinction and explain the difference between one‑dimensional and multidimensional features in the context of SAEs and linear representations.

Here I explain the distinction between one‐dimensional and multidimensional features as they relate to SAEs and linear representations. It uses... (read 547 more words →)

LLMs are really good at k-order thinking (where k is even)

charlieoneill

I've noticed something about how humans and language models work together. There's a pattern that emerges whenever we collaborate effectively.

It goes like this: Someone has an initial idea (step 1). An LLM can then generate variations and connections around that idea (step 2). A human needs to look at these and decide which are actually valuable (step 3). Then the LLM can develop the chosen direction with consistency (step 4).

This alternating pattern shows up everywhere once you start looking for it. The even-numbered steps—expansion, elaboration, systematisation—are what language models do well. The odd-numbered steps—origination, curation, judgement, taste—stay firmly in human hands. In technical terms, LLMs excel at k-order thinking when k is... (read 396 more words →)

Replying toThe nihilism of NeurIPS

charlieoneill1y

The nihilism of NeurIPS

Thank you for laying out a perspective that balances real concerns about misaligned AI with the assurance that our sense of purpose needn’t be at risk. It’s a helpful reminder that human value doesn’t revolve solely around how “useful” we are in a purely economic sense.

If advanced AI really can shoulder the kinds of tasks that drain our energy and attention, we might be able to redirect ourselves toward deeper pursuits—whether that’s creativity, reflection, or genuine care for one another. Of course, this depends on how seriously we approach ethical issues and alignment work; none of these benefits emerge automatically.

I also like your point about how Zen practice emphasises that our humanity isn’t defined by constant production. In a future where machines handle much of what we’ve traditionally laboured over, the task of finding genuine meaning will still be ours.

Replying toThe nihilism of NeurIPS

charlieoneill1y

The nihilism of NeurIPS

You raise a good point: sometimes relentlessly pursuing a single, rigid “point of it all” can end up more misguided than having no formal point at all. In my more optimistic moments, I see a parallel in how scientific inquiry unfolds.

What keeps me from sliding into pure nihilism is the notion that we can hold meaning lightly but still genuinely. We don’t have to decide on a cosmic teleology to care deeply about each other, or to cherish the possibility of building a better future—especially now, as AI’s acceleration broadens our horizons and our worries. Perhaps the real “point” is to keep exploring, keep caring, and keep staying flexible in how we define what we’re doing here.

The nihilism of NeurIPS

charlieoneill

"What is the use of having developed a science well enough to make predictions if, in the end, all we're willing to do is stand around and wait for them to come true?" F. SHERWOOD HOWLAND in his speech accepting the Nobel Prize in Chemistry in 1995.

"Once upon a time on Tralfamadore there were creatures who weren’t anything like machines. They weren’t dependable. They weren’t efficient. They weren’t predictable. They weren’t durable. And these poor creatures were obsessed by the idea that everything that existed had to have a purpose, and that some purposes were higher than others. These creatures spent most of their time trying to find out what their purpose

... (read 1100 more words →)

107

Replying toCan quantised autoencoders find and interpret circuits in language models?

charlieoneill2y

Can quantised autoencoders find and interpret circuits in language models?

I agree - you need to actual measure the specificity and sensitivity of your circuit identification. I'm currently doing this with attention heads specifically, rather than just the layers. However, I will object to the notion of "overfitting" because the VQ-VAE is essentially fully unsupervised - it's not really about the DT overfitting because as long as training and eval error are similar then you are simply looking for codes that distinguish positive from negative examples. If iterating over these codes also finds the circuit responsible for the positive examples, then this isn't overfitting but rather a fortunate case of the codes corresponding highly to the actions of the circuit for the... (read more)

Can quantised autoencoders find and interpret circuits in language models?

charlieoneill

Executive Summary

I try vector-quantised autoencoders (VQ-VAEs) as an alternative compression scheme of transformer activations (as opposed to something like a sparse autoencoder).
- Whilst people have danced around this idea before, discrete quantisation has only ever been tried in the actual transformer architecture itself, rather than on cached activations.
Specifically, I train a VQ-VAE on a data of 1000 cached model activations on the indirect objection identification (IOI) task (500 "positive" examples where the model is required to do IOI, and 500 "negative" examples where it isn't).
- For each forward pass through the model (i.e. for each example), this produces a sequence of n_layers discrete integer codes, supposedly capturing the semantics of that progression of the

... (read 6962 more words →)

Replying toOpen Thread – Winter 2023/2024

charlieoneill2y

Open Thread – Winter 2023/2024

@Ruby @Raemon @RobertM I've had a post waiting to be approved for almost two weeks now (https://www.lesswrong.com/posts/gSfPk8ZPoHe2PJADv/can-quantised-autoencoders-find-and-interpret-circuits-in, username: charlieoneill). Is this normal? Cheers!