Thank you to the AI safety people

juliawise

I was born at the end of the cold war, unaware of the danger we were emerging from. Duck and cover drills (as if those could protect schoolchildren from a nuclear blast) were a curiosity of the past.

The farmer-poet Wendell Berry, who wrote about the dread of nuclear war, died recently. From his 1968 “The peace of wild things”:

When despair for the world grows in me
and I wake in the night at the least sound
in fear of what my life and my children’s lives may be,
I go and lie down where the wood drake
rests in his beauty on the water, and the great heron feeds.
I come into the peace of wild things
who do not tax their lives with forethought
of grief.

*Mid-20th-century fallout shelter, Massachusetts*

I’m so grateful for the generations of people who worked hard to lower our risk from nuclear war. There’s no way to know how close we came to disaster. I’m grateful for the planning, the research, the negotiation, the recognition of common interests, the swallowing of pride. Humanity had built something that could overmaster us all, but people of many countries worked for decades to keep it in check.

And now we’re again building something with great gleaming promise, and also with monstrous potential. As AI development speeds up, I’m afraid for my children, as Wendell Berry was afraid for his.

A lot of people are working hard to help humanity thread the needle. They’re researchers, writers, and policy people. They’re running payroll and conferences and all the other moving parts. They disagree about a lot of things, but they’re willing to work together on a problem bigger than their differences.

I have so much respect and gratitude for all the people working on this, in their different ways. Thank you.

…..
*I don’t know what it’s like to be a wood drake or a heron.

I don’t know what it’s like to be a wood drake or a heron.

What a wonderful excuse to share one of my favourite passages from Wendell Berry's A Native Hill:

I sat one summer evening and watched a great blue heron make his descent from the top of the hill into the valley. He came down at a measured deliberate pace, stately as always, like a dignitary going down a stair. And then, at a point I judged to be midway over the river, without at all varying his wingbeat he did a backward turn in the air, a loop-the-loop. It could only have been a gesture of pure exuberance, of joy — a speaking of his sense of the evening, the day’s fulfillment, his descent homeward. He made just the one slow turn, and then flew on out of sight in the direction of a slew farther down in the bottom. The movement was incredibly beautiful, at once exultant and stately, a benediction on the evening and on the river and on me. It seemed so perfectly to confirm the presence of a free nonhuman joy in the world — a joy I feel a great need to believe in — that I had the skeptic’s impulse to doubt that I had seen it. If I had, I thought, it would be a sign of the presence of something heavenly in the earth.

That is indeed a great opportunity! I don't think I'd read that passage, thanks. I do find it hard to know what to make of people projecting experiences / emotions onto animals; seems pretty likely we're wrong a lot when we do this. "a joy I feel a great need to believe in" feels honest at least about the bias.

At risk of yucking your yum... a lot of the people that want to be described as "helping thread the needle" are also the ones making the danger happen in the first place. I don't feel like the people in the Manhattan Project exploring how to make a nuclear reaction controllable enough to package into a bomb are worthy of congratulations for their efforts in nuclear safety. And I don't know how I feel about prose which encourages us to forget our differences and sing kumbaya. The revealed preference of the builders has not been the safety of humanity, but profit and power, ditching constraints at every turn and ignoring superhuman alignment in favor of short term usability patches. Sentiment like this preferentially advantages their aims.

I did think about writing "Thank you to people doing helpful things (which I'm not sure how to identify) and but not to people doing unhelpful things (which I'm also not sure how to identify)" but this was not that piece.

I don’t know what it’s like to be a wood drake or a heron.

What a wonderful excuse to share one of my favourite passages from Wendell Berry's A Native Hill:

I sat one summer evening and watched a great blue heron make his descent from the top of the hill into the valley. He came down at a measured deliberate pace, stately as always, like a dignitary going down a stair. And then, at a point I judged to be midway over the river, without at all varying his wingbeat he did a backward turn in the air, a loop-the-loop. It could only have been a gesture of pure exuberance, of joy — a speaking of his sense of the evening, the day’s fulfillment, his descent homeward. He made just the one slow turn, and then flew on out of sight in the direction of a slew farther down in the bottom. The movement was incredibly beautiful, at once exultant and stately, a benediction on the evening and on the river and on me. It seemed so perfectly to confirm the presence of a free nonhuman joy in the world — a joy I feel a great need to believe in — that I had the skeptic’s impulse to doubt that I had seen it. If I had, I thought, it would be a sign of the presence of something heavenly in the earth.

56

Thank you to the AI safety people

56

56

56