This was my Moment of Darkness speech for the NYC 2017 Solstice. The overall theme for the event was "generational knowledge."

To pass generational knowledge into the future, you need three things. You need mentors, sharing the most important wisdom they have to offer. You need children, curious and excited to learn, but willing to challenge that wisdom as times change.

Thirdly, you need there to be a future.

Earlier this year I moved to Berkeley, California. Soon afterwards I started hearing news about North Korea's nuclear's testing.

Every few years North Korea talks a big talk about their weapons program, but this year for the first time, we had seismological evidence that they were  testing warheads in excess of 100 kilotons. And that they had intercontinental ballistic missiles that could strike the west coast. And San Francisco was one of the plausible targets.

A friend of mine did some calculations on blast radius. Depending on the exact size of the bomb, they estimated that as long as we stayed indoors, people in Berkeley would probably survive.

This wasn't especially reassuring. 

Now, I have no idea how scared it's actually appropriate to be about this. In politics, and journalism, we're incentivized to stories that freak us out and exaggerate risks.

But one way or another, a lot of things have happened this year that have made it click together for me, that we might not make it. I've understood, intellectually, that humanity barely scrapped by through the cold war, that democracy is precarious. That we can't seem agree on any serious solutions to climate change. That biological terrorism or artificial intelligence might be even more challenging. 

But this is the first year I've felt in my gut, that there are no competent grownups somewhere who know what they're doing and are going to make sure things are okay. 

I've been thinking about that. And I've been thinking about a question that I often get after Solstice.

The question is "What's the deal with the stars?"

The first half of the Solstice is very concrete. We're talking and singing about real history, real people, real problems that they faced. We delve into messy questions...

...and then the second half of the Solstice doesn't answer those questions.

It can't. Because we don't know the answers. Yet.

And even if we could, I wouldn't want to enshrine our best guesses about the future in song.

Instead, we sing about stars that seem unimaginably far away, and a future that maybe we can we make better.

Some people have said this feels like science fiction. Or wish fulfillment. We looked at a messy, broken world, and we couldn't figure out what to do with it. So we took one mythological heaven, and we swapped it out for another of a different genre.

I think that's a fair criticism, and deserves an answer.

Different people in this room have different ways of thinking about the future. We don't agree on how this world is shaped. We don't agree on what the best ways to shape it are, and we don't even agree on how the future should be shaped, even if we knew what we were doing.

So it's hard to tell a concrete story about the future that everyone can believe in.

But there's one thing the ancient astronomers got right, if for the wrong reasons:

One day the sun is going to die. And our little corner of the universe will be plunged into an eternal winter darker than anyone can imagine.

I have no idea what happens in the 5 billion years between now and then. But by the end, I really only see three possibilities:

One: that the children of Earth will have long since perished.

Another: We'll have created systems too powerful to control, that don't understand or don't care about the things we find most precious, and those systems will spread across the universe, creating something hollow at best, horrifying at worst. 

The third possibility is that we got our shit together. 

I'm not sure whether it's harder to imagine 5 billion years, or to imagine humanity getting over our bullshit. On the one hand, 5 billion years is a ludicrously big number. On the other hand, our species is still arguing over things like "is evidence and reason even a good way to understand the world and help people?"

There are problems barreling down at us at 90 miles an hour that we do not yet have the capacity to solve. But if we can get our shit together, if we can solve the problems of the next few centuries, I don't think there'd be anything stopping us. At the very least, the death of the sun would be trivial, one more relatively simple problem to resolve.

It's important to remember that on a geologic timescale, humanity is a child. Or, perhaps a gangly teenager, suddenly finding themselves with limbs longer than they know what to do with. From the perspective of the Earth, most of human history took place in the blink of an eye. It takes evolution a million years to solve problems that we can solve now in a decades or less. 

And while I don't think our current best is good enough, the fact is we have a strong track record of making our best better. When you hear about atrocities and abuses of power on social media, it's important to remind yourself that this isn't because everything is getting worse - it's because our standards have gotten higher. 

Atrocities and abuses of power used to be par for the course. But decade by decade, people have looked at the status quo and said "this is not good enough. This is unacceptable. And I'm going to change it."

Humanity is growing up.

And whenever I start to doubt that, I remember people - some in this room, others across the world, who systemically work to make their best better. Who looked at the distant future they wanted to make reality, and figured out what research needs to be done and which institutions needed to change and which skills they needed to make that happen.

And then they did it. 

And they taught other people to do it. And those people are teaching others in turn.

I don't know whether we'll grow up in time to meet the challenges ahead of us. I don't know what humanity might become if we make it through the era where our power outpaces our wisdom.

But if we do make it, whatever sort of world we chose to build... 

By the time the sun dies, our children will have gone the stars.

New Comment
5 comments, sorted by Click to highlight new comments since:

What behaviors do you change if you take the specific threats more seriously than "wow, scary. but we're getting better in some ways, so we might get better in this way too!"? I came of awareness toward the end of the cold war, and the fear that humanity will self-destruct has been fairly constant for at least the last 40 years. The doomsday clock has been "minutes to midnight" since inception, never "hours".

That it hasn't yet happened does not comfort me, but I'm of the opinion that my influence on the world is better spent on smaller improvements. The filter will happen or it won't, and conditional on it not happening, I can make the results slightly better.

I spent about 5 years assuming I wasn't really a candidate for "major contributor to X-Risk as a cause" except as a donor and maybe community organizer.

I do think it's important that people preserve a healthy psychological option of... just not stressing out too much about this. (One person I respect said "I spent years not focusing on X-risk because I need to recover and focus on my personal life. But I appreciated once-a-year going to Solstice and having the question asked "do you want to make sure the future is okay?")

But, if you're in a place where this at least seems like a reasonable question, here's my personal approach. (If you're more worried about other risks than AI, I think the same framework applies to other areas)

The first two steps of an OODA-loop-like-process (I'm not 100% sure how to separate "observe" and "orient") included:

a) read lots about various x-risks

b) locate people who seemed able to contribute time towards helping me thinking about who weren't too busy.

c) spent 4 hours thinking about AI X-Risk to get a rough orientation of the situation, and what options seemed available to me.

(I think it's worth doing the above even after reading the results of other people doing the above, since your situation and theirs may be different)

The main things this output were:

  1. Decide if I had the financial and emotional slack to take on anything serious here. (I did, and if I hadn't, I'd have focused more on getting a good job to build more financial stability. Or, potentially focused on earning-to-give and then setting up a monthly donation to orgs that seem important. "Monthly donation" being important since it lets them plan for the future better.)
  2. Generally focus on building habits that increase the range of my abilities, opening more options up.
  3. Change my environment and network such that I could more easily notice opportunities to help and do them", followed by another Observe/Orient loop.

I'm getting towards the "decide" section of the next OODA cycle, and current considerations include:

1) Continuing to work on LesserWrong, after being convinced by the argument "Less Wrong was one of the few reliable pipelines outputing more AI safety researchers, and making sure that it is still thriving is a good way to do that."

2) Spending at least a week trying to learn Machine Learning tools that may enable to help with some technical research projects. (My current understanding is that there's room there for people who are technically competent but not exceptional, although it may requires some particular skillsets which may or may not come naturally to me)

3) Contribute organizational capacity to other new projects that look like they could use help getting off the ground, and/or gain managerial skills that I can use to help flesh out the hierarchies of other organizations.

At our Berlin Solstice event we sung Brighter than Today and played in the background. We noticed that the complexity of that recording is higher than the version that was actually song at the last solstice for which there's a public recording of the whole solstice event.

It would be great to have the a youtube version of the sing along version (at best with the song text).

Apologies for this. The issue is that getting a professional recording is a lot of work, and the existing new recordings are somewhat sketchy live recordings that I feel awkward about charging money for on bandcamp and would feel slightly weird to sing along to on youtube.

I have the 2016 version here on soundcloud, where I put my unfinished works. (This is the final canonical version, although this recording has some mistakes)

I also have a version that sounds good... but uses an unauthorized background instrumental track I didn't create myself. I tried to get permission to use it but the creator didn't let me. I think it's fine to pass around unofficially but not to put on youtube or to use at events that charge money. (I actually paid for the rights to use the instrumentals in "a movie" but they specifically denied authorization to use it to create a formal new song). A version with me singing-of-moderate-quality (but with weird cars driving by in the background) is here. A purely instrumental version is here.

"Now, I have no idea how scared it's actually appropriate to be about this. In politics, and journalism, we're incentivized to stories that freak us out and exaggerate risks."

I'm struggling with exactly this right now regarding the North Korea stuff. When I'm scared, I can't tell if I'm overreacting. When I'm not scared, I can't tell if I'm underreacting. There are precautions I could be taking to make my own survival more likely in the event of a bombing, but those precautions would be a big waste of resources if they aren't needed. The people around me aren't freaking out, [but that doesn't comfort me.](