Survival without dignity

[-]Cole Wyeth1y6625

I think this is a story about anthropic immortality.

[-]Linch1y30

Yeah this was my first thought halfway through. Way too many specific coincidences to be anything else.

[-]L Rudolf L1y20

I did not actually consider this, but that is a very reasonable interpretation!

(I vaguely remember reading some description of explicitly flat-out anthropic immortality saving the day, but I can't seem to find it again now)

[-]Viktor Rehnberg1y22

(Perhaps you're thinking of this https://www.lesswrong.com/posts/EKu66pFKDHFYPaZ6q/the-hero-with-a-thousand-chances)

[-]Linch1y30

There's also a comic series with explicitly this premise, unfortunately this is a major plot point so revealing it will be a spoiler:

Transdimensional Brain-Chip

[-]Tapatakt1y1917

I often think something like "It's a shame there's so little modern science fiction that takes AI developments into account". Thanks for doing something in this niche, even if in such small form!

[-]L Rudolf L1y92

Agreed! Transformative AI is hard to visualise, and concrete stories / scenarios feel very lacking (in both disasters and positive visions, but especially in positive visions).

I like when people try to do this - for example, Richard Ngo has a bunch here, and Daniel Kokotajlo has his near-prophetic scenario here. I've previously tried to do it here (going out with a whimper leading to Bostrom's "disneyland without children" is one of the most poetic disasters imaginable - great setting for a story), and have a bunch more ideas I hope to get to.

But overall: the LessWrong bubble has a high emphasis on radical AI futures, and an enormous amount of fiction in its canon (HPMOR, Unsong, Planecrash). I keep being surprised that so few people combine those things.

[-]Tapatakt1y40

I guess the big problem for someone who tries to do it not in small form is that while you write the story it is already getting old. There are writers who can write a novel in a season, but not many. At least if we talk about good writers. Hm-m-m, did rationalists try to hire Stephen King? :)

[-]L Rudolf L1y40

Do the stories get old? If it's trying to be about near-future AI, maybe the state-of-the-art will just obsolete it. But that won't make it bad necessarily, and there are many other settings than 2026. If it's about radical futures with Dyson spheres or whatever, that seems like at least a 2030s thing, and you can easily write a novel before then.

Also, I think it is actually possible to write pretty fast. 2k/day is doable, which gets you a good length novel in 50 days; even x3 for ideation beforehand and revising after the first draft only gets you to 150 days. You'd have to be good at fiction beforehand, and have existing concepts to draw on in your head though

[-]gabrielrecc1y189

Love pieces that manage to be both funny and thought-provoking. And +1 for fitting a solar storm in there. There is now better evidence of very large historical solar storms than there had been during David Roodman's Open Phil review in late 2014, have been meaning to write something up about that but other things have taken priority.

[-]NunoSempere1y60

I have a writeup on solar storm risk here that could be of interest

[-]Thomas Kwa1y*101

In many respects, I expect this to be closer to what actually happens than "everyone falls over dead in the same second" or "we definitively solve value alignment". Multipolar worlds, AI that generally follows the law (when operators want it to, and modulo an increasing number of loopholes) but cannot fully be trusted, and generally muddling through are the default future. I'm hoping we don't get instrumental survival drives though.

[-]Donald Hobson1y73

The "Warring nanobots in the upper atmosphere" thing doesn't actually make sense.

The zaps of light are diffraction limited. And targeting at that distance is hard. Partly because it's hard to tell between an actual animal and a bunch of nanobots pretending to be an animal. So you can't zap the nanobots on the ground without making the ground uninhabitable for humans.

The "California red tape" thing implies some alignment strategy that stuck the AI to obey the law, and didn't go too insanely wrong despite a superintelligence looking for loopholes (Eg the social persuasion infrastructure is already there. Convince humans that dyson sphere are pretty and don't block the view?).

There is also no clear explanation of why someone somewhere doesn't make a non-red-taped AI.

[-]L Rudolf L1y116

You have restored my faith in LessWrong! I was getting worried that despite 200+ karma and 20+ comments, no one had actually nitpicked the descriptions of what actually happens.

The zaps of light are diffraction limited.

In practice, if you want the atmospheric nanobots to zap stuff, you'll need to do some complicated mirroring because you need to divert sunlight. And it's not one contiguous mirror but lots of small ones. But I think we can still model this as basic diffraction with some circular mirror / lens.

Intensity , where $E$ is the total power of sunlight falling on the mirror disk, $r$ is the radius of the Airy disk, and $c_{e}$ is an efficiency constant I've thrown in (because of things like atmospheric absorption (Claude says, somewhat surprisingly, this shouldn't be ridiculuously large), and not all the energy in the diffraction pattern being in the Airy disk (about 84% is, says Claude), etc.)

Now, $E = π {(\frac{D}{2})}^{2} L$ , where $D$ is the diameter of the mirror configuration, $L$ is the solar irradiance. And $r = θ l$ , where $l$ is the focal length (distance from mirror to target), and $θ \approx 1.22 λ / D$ the angular size of the central spot.

So we have $I \approx \frac{c_{e} L D^{4}}{{1.22}^{2} \times 4 λ^{2} l^{2}}$ , so the required mirror configuration radius $D = \sqrt[4]{\frac{{1.22}^{2} \times 4 I λ^{2} l^{2}}{c_{e} L}}$ .

Plugging in some reasonable values like $λ \approx 5 \times 10^{- 7}$ m (average incoming sunlight - yes the concentration suffers a bit because it's not all this wavelength), $I = 10^{7}$ W/m^2 (the level of an industrial laser that can cut metal), $l = 10^{4}$ m (lower stratosphere), $L = 1361$ W/m^2 (solar irradiance), and a conservative guess that 99% of power is wasted so $c_{e} = 0.01$ , we get $D \approx 18$ m (and the resulting beam is about 3mm wide).

So a few dozen metres of upper atmosphere nanobots should actually give you a pretty ridiculous concentration of power!

(I did not know this when I wrote the story; I am quite surprised the required radius is this ridiculously tiny. But I had heard of the concept of a "weather machine" like this from the book Where is my flying car?, which I've reviewed here, which suggests that this is possible.)

Partly because it's hard to tell between an actual animal and a bunch of nanobots pretending to be an animal. So you can't zap the nanobots on the ground without making the ground uninhabitable for humans.

I don't really buy this, why is it obvious the nanobots could pretend to be an animal so well that it's indistinguishable? Or why would targeted zaps have bad side-effects?

The "California red tape" thing implies some alignment strategy that stuck the AI to obey the law, and didn't go too insanely wrong despite a superintelligence looking for loopholes

Yeah, successful alignment to legal compliance was established without any real justification halfway through. (How to do this is currently an open technical problem, which, alas, I did not manage to solve for my satirical short story.)

Convince humans that dyson sphere are pretty and don't block the view?

This is a good point, especially since high levels of emotional manipulation was an established in-universe AI capability. (The issue described with the Dyson sphere was less that it itself would block the view, and more that building it would require dismantling the planets in a way that ruins the view - though now I'm realising that "if the sun on Earth is blocked, all Earthly views are gone" is a simpler reason and removes the need for building anything on the other planets at all.)

There is also no clear explanation of why someone somewhere doesn't make a non-red-taped AI.

Yep, this is a plot hole.

[-]Donald Hobson1y50

why is it obvious the nanobots could pretend to be an animal so well that it's indistinguishable?

These nanobots are in the upper atmosphere, possibly with clouds in the way, and the nanobot fake humans could be any human to nanobot ratio. Nanobot internals except human skin and muscles. Or just a human with a few nanobots in their blood.

Or why would targeted zaps have bad side-effects?

Because nanobots can be like a bacteria if they want. Tiny and everywhere. The nanobots can be hiding under leaves, cloths, skin, roofs etc. And even if they weren't, a single nanobot is a tiny target. Most of the energy of the zap can't hit a single nanobot. Any zap of light that can stop nanobots in your house needs to be powerful enough to burn a hole in your roof.

And even if the zap isn't huge, it's not 1 or 2 zapps, it's loads of zapps constantly.

[-]Martin Vlach1y60

some OpenAI board members who the Office of National AI Strategy was allowed to appoint, and they did in fact try to fire Sam Altman over the UAE move, but somehow a week later Sam was running the Multinational Artificial Narrow Intelligence Alignment Consortium, which sort of morphed into OpenAI's oversight body, which sort of morphed into OpenAI's parent company, and, well, you can guess who was running that.

pretty sassy abbreviations spiced in there.'Đ

I've expected the hint of
> My name is Anthony. What would you like to ask?
to show it Anthony was an LLM-based android, but who knows.?.

[-]TropicalFruit11mo50

“Twitter, I mean X” in 2045 had me dying. So did the California red tape part. Awesome job.

[-]Noosphere891y50

I think this is a story about how AI was severely regulated by the courts, leading to incentives away from AI progress combined with global catastrophic events setting humanity's technology back by a lot, so AIs didn't kill us.

[-]Ann1y50

It's nice to read some realistic science fiction.

[-]Ann1y20

Too much runs into the very real issue that truth is stranger. 😉

[-]KatWoods10mo40

I laughed out loud so many times reading this. Thank you for writing it.

[-]Christo Wilken1y42

This was wonderful. Anyone have recommendations for similar writing?

[-]Neil1y111

Here's 4:

https://www.lesswrong.com/posts/kj4jW9DxtKQBJbapn/stanislav-petrov-quarterly-performance-review

https://www.astralcodexten.com/p/every-bay-area-house-party?utm_campaign=posts-open-in-app&triedRedirect=true

https://www.lesswrong.com/posts/pL4WhsoPJwauRYkeK/moses-and-the-class-struggle

https://www.lesswrong.com/posts/aRBAhBsc6vZs3WviL/ommc-announces-rip

[-]L Rudolf L1y134

Good list!

I personally really like Scott Alexander's Presidential Platform, it hits the hilarious-but-also-almost-works spot so perfectly. He also has many Bay Area house party stories in addition to the one you link (you can find a bunch (all?) linked at the top of this post). He also has this one from a long time ago, which has one of the best punchlines I've read.