LESSWRONG
LW

Maxwell Clarke — LessWrong

I think a lot of people can't think at the right level of abstraction for understanding Yudkowsky. Some things are overdetermined because of the high level structure of reality. Start with physical reality as we best understand it, then derive what is possible eventually, then derive what is possible soon, then derive the incentives, then derive categories of what will happen. This completely top-down way of drawing conclusions is perhaps tricky to get right, but gives broad predictions far into the future. At no point till now have any facts about AI development contradicted Yudkowsky and Bostrom's arguments from this basis. At no point do these arguments rely on anything one has... (read more)

Maxwell Clarke's Shortform

Maxwell Clarke

6mo

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

Maxwell Clarke6moQuick Take

Pre-training vs Reinforcement Learning

Pre-training

In pre-training, the model learns to mirror a dataset.

Pre-training trains a model on a dataset - the model is trained to align with a set of tokens/actions, but the model never actually outputs any itself. Any biases (vs. the dataset) in the model towards one action/output sequence or the other cause poorer performance, and the model comes out relatively unbiased (vs. the dataset) as a result. In other words, the model comes out as a close representation of the dataset.

At inference time, the model can be used to produce actions/output sequences - it will then receive its own outputs as input - and any bias in the model can... (read 780 more words →)

Maxwell Clarke3y

In NZ we have biting bugs called sandflies which don't do this - you can often tell the moment they get you.

Replying toNo - AI is just as energy-efficient as your brain.

Maxwell Clarke3y

No - AI is just as energy-efficient as your brain.

Yes, that's fair. I was ignoring scale but you're right that it's a better comparison if it is between a marginal new human and a marginal new AI.

Replying toNo - AI is just as energy-efficient as your brain.

Maxwell Clarke3y

No - AI is just as energy-efficient as your brain.

Well, yes, the point of my post is just to point out that the number that actually matters is the end-to-end energy efficiency — and it is completely comparable to humans.

The per-flop efficiency is obviously worse. But, that's irrelevant if AI is already cheaper for a given task in real terms.

I admit the title is a little clickbaity but i am responding to a real argument (that humans are still "superior" to AI because the brain is more thermodynamically efficient per-flop)

Replying toNo - AI is just as energy-efficient as your brain.

Maxwell Clarke3y

No - AI is just as energy-efficient as your brain.

I saw some numbers for algae being 1-2% efficient but it was for biomass rather than dietary energy. Even if you put the brain in the same organism, you wouldn't expect as good efficiency as that. The difference is that creating biomass (which is mostly long chains of glucose) is the first step, and then the brain must use the glucose, which is a second lossy step.

But I mean there is definitely far-future biopunk options eg. I'd guess it's easy to create some kind of solar panel organism which grows silicon crystals instead of using chlorophyll.

No - AI is just as energy-efficient as your brain.

Maxwell Clarke

This is simply because AIs run on electricity.

Our brains use sunlight via photosynthesis via dietary energy intake. From sunlight to dietary energy this is about 0.25-0.5% energy efficient. Let's say it's 0.35% efficient. (This is then our complete sunlight-to-capability efficiency for a human)

AI systems use sunlight via solar energy via electrical consumption. From sunlight to GPU this is about 10-18% efficient let's say it's 13% efficient. (Data: typical solar panel efficiency = 15-20% efficient. Typical electricity distribution efficiency = 90% efficient). 13% is then AI sunlight-to-electricity efficiency, but not yet sunlight-to-capability efficiency.

Let's calculate the final part. We need to assume an amount of compute for a human-equivalent AI system. I will assume... (read more)

Replying toModels Don't "Get Reward"

Maxwell Clarke3y

Models Don't "Get Reward"

Fully agree - if the dog were only trying to get biscuits, it wouldn't continue to sit later on in it's life when you are no longer rewarding that behavior.Training dogs is actually some mix of the dog consciously expecting a biscuit, and raw updating on the actions previously taken.

Hear sit -> Get biscuit -> feel good
becomes
Hear sit -> Feel good -> get biscuit -> feel good
becomes
Hear sit -> feel good
At which point the dog likes sitting, it even reinforces itself, you can stop giving biscuits and start training something else

Replying toCategorizing failures as “outer” or “inner” misalignment is often confused

Maxwell Clarke3y

Categorizing failures as “outer” or “inner” misalignment is often confused

This is a good post, definitely shows that these concepts are confused. In a sense both examples are failures of both inner and outer alignment -

Training the AI with reinforcement learning is a failure of outer alignment, because it does not provide enough information to fully specify the goal.
The model develops within the possibilities allowed by the under-specified goal, and has behaviours misaligned with the goal we intended.

Also, the choice to train the AI on pull requests at all is in a sense an outer alignment failure.

Replying toExploring Mild Behaviour in Embedded Agents

Maxwell Clarke3y

Exploring Mild Behaviour in Embedded Agents

If we could use negentropy as a cost, rather than computation time or energy use, then the system would be genuinely bounded.

Replying toA Mystery About High Dimensional Concept Encoding

Maxwell Clarke3y

A Mystery About High Dimensional Concept Encoding

Gender seems unusually likely to have many connotations & thus redundant representations in the model. What if you try testing some information the model has inferred, but which is only ever used for one binary query? Something where the model starts off not representing that thing, then if it represents it perfectly it will only ever change one type of thing. Like idk, whether or not the text is British or American English? Although that probably has some other connotations. Or whether or not the form of some word (lead or lead) is a verb or a noun.

Agree that gender is a more useful example, just not one tha necessarily provides clarity.

My tentative interpretability research agenda - topology matching.

Maxwell Clarke

I'm looking for feedback on these ideas before I start working on them in November.

The goal of interpretability is to understand representations, and to be able to translate between them. This idea tries to discover translations between the representations of two neural networks, but without necessarily discovering a translation into our representations.

TL;DR

Learn auto-encoders in such a way that we know the learned topology of the latent space. Use those topologies to relate the latent spaces of two or more such auto-encoders, and derive (linear?) transformations between the latent spaces. This should work without assuming any kind of privileged basis.

My biggest uncertainty is about whether any of my various approaches to not constraining... (read 1000 more words →)