Eliezer Yudkowsky


The Bayesian Conspiracy
Three Worlds Collide
Highly Advanced Epistemology 101 for Beginners
Inadequate Equilibria
The Craft and the Community
Challenging the Difficult
Yudkowsky's Coming of Age
Quantified Humanism
Value Theory
Load More (9/36)

Wiki Contributions


100 Years Of Existential Risk

It is written nowhere in That Alien Message that the humans do anything human!mean to the aliens in the universe they break out into.  The error strikes me as very significant; to write that humans would be ungracious in victory from a human perspective, be cruel or even uncaring towards the aliens, would indicate that the author's fear of AI came from a belief that everybody has to be uncaring and humans have to be uncaring too, which is very much not what I believe.

(I do expect that the humans stopped the aliens from eating babies; and that the aliens were very sad about that, and that it was not what they'd have chosen for themselves.)

Will Capabilities Generalise More?

Reality hits back on the models we train via loss functions based on reality-generated data. But alignment also hits back on models we train, because we also use loss functions (based on preference data). These seem to be symmetrically powerful forces.

Alignment doesn't hit back, the loss function hits back and the loss function doesn't capture what you really want (eg because killing the humans and taking control of a reward button will max reward, deceiving human raters will increase ratings, etc).  If what we wanted was exactly captured in a loss function, alignment would be easier.  Not easy because outer optimization doesn't create good inner alignment, but easier than the present case.

It’s Probably Not Lithium

Actually I'd ask about the effect of free-feeding non-domesticated animals on ecologically realistic food, rather than free-feeding cows bred to gain weight using grains.

It’s Probably Not Lithium

Yeah, that sounds right - with a non-broken metabolism, eating lots and lots of tasty food that's just prepared and sitting there, to your heart's content, should totally result in about 4 pounds of weight gain, all the way up to 150 pounds.

That's how everybody's metabolisms used to work.

It’s Probably Not Lithium

I hear that "pregnant" people also do less mountain-climbing, even if they were exercising healthily before.  No wonder they gain weight!  Do we even need to postulate "pregnancy" as a condition, when their caloric intake and reduced exercise seems adequate to explain all of the observed weight gain?

It’s Probably Not Lithium

Retrying again:

By the same reasoning:

"Pregnancy" probably isn't a thing.  "Pregnant" people eat around 500 more calories per day.  This is sufficient to explain all the weight gain from "pregnancy" without supposing anything other than thermodynamics at work - anyone who eats an extra 500 calories per day will probably gain that much weight over the course of 40 weeks.

Looking back on my alignment PhD

That does sound like you might've made some progress in the Way, at least; well done on having acquired that much more dignity.

It’s Probably Not Lithium

I'll try rephrasing, somewhat overstating the strength of the reasoning (taking it from probabilistic to logical) in case that causes a basic idea to be clearly communicated that was obscured by probabilism.

If an increase in calorie intake was itself sufficient to produce an increase in fat mass in otherwise metabolically high-functioning adults, we'd have seen a different result from pre-obesity-epidemic experiments in overfeeding.  This rules out the direction of causality "mere overeating" -> "obesity".

Any hypertrophy of fat cells, in turn, will causally require more nutritional intake to feed the hypertrophy, just like a cancer growing, or any other body part growing, will require more nutritional intake.  So once you observe a cancer or a huge fat cell mass or any other diseased body part growing, you already know the person already took in that much food, both to grow the cancerous body part and sustain it; you shouldn't be surprised to look back at the cancer patient's caloric consumption record and find an excess; so finding that excess consumption shouldn't update you at all about the cause of the malevolently growing body part.

You're taking causality that must at least run from "fat cell hypertrophy" -> "excess consumption" and already fully explains away the presence of an observed correlation, and then adding on a causal postulate that runs the other way - a direction of causality that would be theoretically possible in a world with no overfeeding experiments one way or the other, though not supported even there, since there is no otherwise unexpected observation which it explains; but which in our world is ruled out by the results of overfeeding experiments, which tested the results of experimental-intervention-produced excess calorie consumptions in metabolically healthy individuals before the obesity epidemic.

To further oversimplify the oversimplification: the logic you're deploying for obesity would also work to conclude that overeating causes cancer, and therefore Proves Too Much.

The only reason why "overeating caused this huge fat mass to grow inside my body" sounds more plausible than "overeating caused this huge tumor to grow inside my body" is that the former theory follows the Sin Theory of Obesity in which obesity is a punishment for the sin of gluttony, while the latter theory is incongruent with simple just-world hypotheses as of the 21st century in Western societies.  Both are ruled out by experiments showing that (in metabolically healthy individuals before the obesity epidemic) a randomized experimental intervention to add overeating does not produce obesity any more than it produces tumors.

It’s Probably Not Lithium

Adderall worked for you.  It didn't work for me.

Load More