x

LESSWRONG

LW

Charlie Steiner — LessWrong

Charlie Steiner

Top postsTop post

Charlie Steiner

Message

If you want to chat, message me!

LW1.0 username Manfred. PhD in condensed matter physics. I am independently thinking and writing about value learning.

8796

Ω

1218

77

2457

9y

Charlie Steiner

If you want to chat, message me!

LW1.0 username Manfred. PhD in condensed matter physics. I am independently thinking and writing about value learning.

Top postsTop post

Some background for reasoning about dual-use alignment research

This is pretty basic. But I still made a bunch of mistakes when writing this, so maybe it's worth writing. This is background to a specific case I'll put in the next post. It's like a a tech tree If we're looking at the big picture, then whether some piece of research is net positive or net negative isn't an inherent property of that research; it depends on how that research is situated in the research ecosystem that will eventually develop superintelligent AI. A tech tree, with progress going left to right. Blue research is academic, green makes you money, red is a bad ending, yellow is a good ending. Stronger connections are more important prerequisites. Consider this toy game in the picture. We start at the left and can unlock technologies, with unlocks going faster the stronger our connections to prerequisites. The red and yellow technologies in the picture are superintelligent AI - pretend that as soon as one of those technologies is unlocked, the hastiest fraction of AI researchers are immediately going to start building it. Your goal is for humanity to unlock yellow technology before a red one. This game would be trivial if everyone agreed with you. But there are many people doing research, and they have all kinds of motivations - some want as many nodes to be unlocked as possible (pure research - blue), some want to personally unlock a green node (profit - green), some want to unlock the nearest red or yellow node no matter which it is (blind haste - red), and some want the same thing as you (beneficial AI - yellow) but you have a hard time coordinating with them. In this baseline tech tree game, it's pretty easy to play well. If you're strong, just take the shortest path to a yellow node that doesn't pass too close to any red nodes. If you're weak, identify where the dominant paradigm is likely to end up, and do research that differentially advantages yellow nodes in that future. The tech tree is wrinkly But of course there are lots of wrinkles not i

127May 18, 2023

Neural uncertainty estimation review article (for alignment)

How to turn money into AI safety?

Training on probes: What's going on

Training on probes: Research ideas

Recap Sequel to Previous Post. This post might not make sense without it. Last post, I told some stories about how training on probes might let our judgments on easy domains generalize to harder domains by leveraging an AI's model of the world. Training using the gradient of the probe...

Training on probes: What's going on

TL;DR * If you train a probe for some property (like "honesty") and do gradient descent against this probe while continuing training that incentivizes dishonesty, the model will change its internal representation to evade the probe. Duh. * If you train against the probe after all other training, it works...

I (well, mostly claude code) simulated proportional representation methods.

Low-ish effort post just sharing something I found fun. No AI-written text outside the figures. I was recently nerd-sniped by proportional representation voting, and so when playing around with claude code I decided to have it build a simulation. Hot take: * If you're electing a legislature and want it...

Low-effort review of "AI For Humanity"

Stumbled across a book in the new section of the library: "AI For Humanity," by Andeed Ma, James Ong (founder of the think tank AIII, which is also the sound I make when thinking about AI risk), and Siok Siok Tan. It's a mass-market-ish book about, well, AI for humanity,...

Dec 11, 2024•14

Rabin's Paradox

Quick psychology experiment Right now, if I offered you a bet[1] that was a fair coin flip, on tails you give me $100, heads I give you $110, would you take it? Got an answer? Good. Hover over the spoiler to see what other people think: About 90% of undergrads...

Aug 14, 2024•19

Humans aren't fleeb.

In the oceans of the planet Water, a species of intelligent squid-like aliens - we'll just call them the People - debate about what it means to be fleeb. Fleeb is a property of great interest to the People, or at least they think so, but they also have a...

Jan 24, 2024•38

Neural uncertainty estimation review article (for alignment)

EDIT 1/27: This post neglects the entire sub-field of estimating uncertainty of learned representations, as in https://openreview.net/pdf?id=e9n4JjkmXZ. I might give that a separate follow-up post. Introduction Suppose you've built some AI model of human values. You input a situation, and it spits out a goodness rating. You might want to...

Dec 5, 2023•74

Load More (7/75)