The Unreasonable Effectiveness of Fiction

[-]cousin_it5h30

My favorite example of fiction influencing reality (or maybe just predicting it really well, it's hard to tell) is how Arthur Conan Doyle's detective stories basically created forensic science from thin air. For example, the very first Holmes story "A Study in Scarlet", published in 1887, describes Holmes inventing a chemical test to distinguish dried bloodstains from dirt stains. Then exactly that test was invented in 1900. Another example is analysis of tiny differences between typewriters, which appeared in Holmes stories a few years before anyone did it in reality.

[-]Darklight10h30

I'm someone with an AI research/engineering background who also aspires to be (and sometimes fancies himself) a good writer. How would you be able to tell if I should put in the time and energy required to write short stories or novels that try to input good sci-fi ideas into our culture (particularly AI safety related) rather than using that time and energy on, for instance, side projects in technical AI safety? It may not be an either/or thing, but I'm not sure splitting my time is better than focusing on one path.

Also, from my cursory research, it seems like becoming a successful published author is similar to winning the lottery, and has worse odds than succeeding at a startup. Would it still make sense to try this, even if realistically, it probably wouldn't be financially sustainable? The hypothetical EV could be very high, but that seems to depend on having very good ideas, very good writing ability, and a certain amount of luck, which are things I'm not super confident I have enough of here.

[-]Raelifin9h20

Yeah, these are good questions. I mostly don't suggest people try to support themselves writing unless they already know they're very good at storytelling, and even then it's hard/rare. Instead, I think it's good for people to experiment with it as a side-thing, ideally in addition to some useful technical work. (I'm very blessed that I get to work as a researcher at MIRI, for example, and then go home and write stories that are inspired by my research.) Don't wait to be discovered by a literary agent; if you write something good, post it online! Only try to seriously monetize after you already have some success.

Regarding how to tell if your stories are good, I think the main thing is to get them in front of people who will be blunt, and find out what they say. LLMs are a good stepping-stone to this, if you're hesitant to get a real human to read your work, though you'll have to shape their prompt so that they're critical and not sycophantic. Writing groups can also be a good resource for testing yourself.

AI in Particular

Appreciating the power of fiction is an important general lesson, but I admit that I have a particular interest in it on the topic of artificial intelligence. When I first started writing about AI, the notion that someone could have a natural-language conversation with their computer was clear science-fiction. It is now science-fact. And yet, there are approximately zero stories about AI that are grounded in the technology that now exists. The few pieces of modern fiction that have come out in this new LLM-age are mostly still mired in old views, and are usually, at best, "the AI acts basically like a weird human."

AIs have texture. They have dynamics and details. They are mysteries to most people. Even those who frequently use chatbots often have little understanding of what is going on behind the scenes, what is possible now, or what might be possible in just a couple years. The world is ripe for a truly great piece of AI fiction that educates as much as it entertains, and gets the world to really grapple with the presence of these new minds.

Is my new novel that story? Probably not. But I'm trying. And I think it's worth thinking about whether you (if you have some combination of storytelling skill and technical insight) ought to be trying too. If we don't find something soon, we run the risk of having the next big, break-out AI story be about hippy robots saving humanity from dehydration by shutting down their factories and wandering off into the woods.^[5]

What's Going On Here?

Of course, not everyone is driven by fiction. For every person who says they chose their career because they resonated with a story, there will be several others who chose their field because of real-world inspirations, pragmatism, or random opportunity. And for those who haven't had their lives and worldviews shaped by fiction, this can all seem overblown. Perhaps my enjoyment of Dune and my affinity for transhumanism are both downstream of some genetic disposition. Perhaps Reagan's interest in the Strategic Defense Initiative was on its merits, and the connection to Murder in the Air is a coincidence.

To bolster the large number of self-reports, I want to present a collection of a priori reasons to expect fiction to change people's minds in ways that can often be more potent than non-fiction:

The Conjunction Fallacy — Humans can infamously give more credence to specific, detailed statements, rather than more general descriptions, even when the general description strictly covers a superset of the detailed one. This bias probably stems from multiple factors, but one likely factor is that detailed stories (that feel real) are more likely to come from a mind that has engaged with the surprisingly detailed real world. If a story lacks details, one implicitly reasons, it is likely fabricated, or at least coming from someone who doesn't know very much about the domain. Well-written fiction exploits this bias, carrying an implicit feeling of reality even when it's explicitly flagged as untrue.^[6]

Less Defensiveness and Backfiring — One of my favorite things, as an author, is writing characters who have a worldview that I disagree with. In my new novel, Red Heart, for instance, I got to write several characters who were various flavors of Chinese communists. Writing these characters, I get to explore new perspectives and try on ideas that I normally reject. I don't need to yell at the page to debunk Chinese communism; they're "just characters in a novel," after all. If they're wrong, it's often assumed that readers will see that. By contrast, nonfiction can backfire, convincing readers that the author is an intellectual opponent that needs to be argued against, either to themselves or on the internet or whatever. People are significantly less identitarian/political when deciding which stories to consume.^[7]

Deep Attention — Stories are entertainment. They are crafted, first and foremost, to be engaging and hold the reader's attention. 300 pages of thriller novel can fly by in a weekend, while most people never read 300 consecutive pages of dry non-fiction in their entire life. I've talked to many laypeople who read all the way through AI 2027, but who never would've found the motivation to read a technical paper half as long. By shaping themselves to the easiest form-factor, fiction authors thus have an unrivaled ability to get readers to attend to a set of ideas for a long period of time, giving those readers the space to think deeply and resist the storm of omnipresent distractions.

Authorial Freedom — Reality is full of inconvenient facts! If you want to argue for something, it's often much easier to contrive of a situation where the world is conveniently set up to showcase the thing you want to focus on. We can see this in action in theoretical physics, where fictional thought experiments are constructed to highlight and demonstrate important principles that would be easy to dismiss in a real-world context. Did Putin rise to power by orchestrating bombings against the Russian people, or is this just a conspiracy theory? Did covid escape from a lab? Are some modern LLMs actually power-hungry, or are they just roleplaying? Irrelevant! Just treat the issues as a given fact within the plot of your story and you can freely explore the consequences.

The Power of Vibes and Memes — Humans are compelled by many things. Rational arguments based on data have some degree of effectiveness on some people... but far, far more people are motivated by how things feel. What's cool? What's exciting? What's hilarious? No statistical analysis will ever come close to the memorability or virality of a good joke. Stories speak primarily to the parts of the mind that listen. Is it any wonder that they can have such a strong impact?

Authorial Responsibility

Unfortunately, with the possible exception of decreased defensiveness, all of the dynamics discussed above are dual-use. For every beautiful story that brings the reader into contact with an important truth, there is at least one other, almost as beautiful, that risks infecting them with falsehood. When you stop being beholden to facts, you run the risk of optimizing too much for persuasion, and not enough for rigor.

This risk is most pronounced when it's woven into the background, or presented as serious physics in domains where people don't have a lot of first-hand experience. Magic and unrealistic physics are fun! I am not demanding that every story with dragons have a plausible evolutionary history where the caloric expenditure of breathing fire makes sense, or that every spaceship take years to travel between the stars. But still, thanks to TV and movies, many people don't realize that that getting hit with a bullet causes almost no knockback, that guns with "silencers" are still loud, and that firing a gun into the air can be deadly.^[8]

The problem comes when the unrealistic bits of a story aren't recognized as fictional by the audience, either because the author was mistaken about something, or because they failed to flag things clearly. This can be particularly pernicious in the domains of psychology, sociology, and economics, where subtle misconceptions and cognitive biases are passed from author to reader. Every plucky rebel is fighting a tyrannical empire. Every environmental catastrophe is the result of corporate greed. Everyone who goes to court is either cleanly innocent or obviously guilty. Humans want things to make sense, and for the events in their lives, big and small, to happen for narratively satisfying reasons. Each story to casually^[9] discard the messy complexity of the world contributes to the problem.

The responsibility thus falls on fiction writers to not just debias ourselves, but to debias our work—to give the reader vicarious experiences that don't just feel real, but which lead them to be better at predicting the true reality. A good story can wake the President of the United States up to technological risks, inspire generations, and clarify complex issues. And just as the risks of misconceptions are high in areas where readers aren't knowledgeable enough to disbelieve, the opportunities to educate are also greatest on the cutting edge. For anyone working with ideas that aren't yet widely understood—whether you're a novelist, a scientist, or simply someone who sees something others have missed—storytelling offers the rare opportunity to draw attention, get people to think deeply, and help them find a hold on the truth, before a more careless writer leads them to falsehood.^[10]

This can be a heavy task. Writing fiction that makes sense and conforms to the laws of economics and physics can rub against the expectations or desires of the audience. But it can also be extremely satisfying when done well. With skill, an active effort to combat the failure modes, and some luck, it is possible to write stories that are effective at getting readers to think and learn about important topics such as AI—perhaps unreasonably effective—but without betraying reason.

^{^}

I'll leave it to readers to decide whether he's living up to the source material.

^{^}

Is this nominative determinism or anti-nominative determinism?

^{^}

He ultimately wound up working in tech because that's where the good jobs were, but the influence was still clearly there. I know many similar instances of people whose career goals were influenced by detective and crime-scene investigation shows, only to have life kick them down a different path. My claim is that fiction is influential, not that its influence is somehow always able to win out over economics.

^{^}

I am personally skeptical of this, and could see it going either way. I came to the field from stumbling on Eliezer's technical work, and don't feel like there's an obvious skew either way, when people tell me how they got involved.

^{^}

People seem to love the "robot connects with nature" trope. And while I think it's basically as bad as "robot melts down when presented with paradox," I want to be clear that I actually think both A Psalm for the Wild-Built and The Wild Robot are pretty good stories. Part of the point of this essay is that a story can manage to be pretty darn good, without being a very good guide to reality.

^{^}

One of my most mortifying memories is from when I met a high-status person for the first time (at a party) and got into a conversation where I boldly claimed that the Disney corporation didn't actually own Disney World. This, to be clear, is extremely obviously false, and I definitely should have known better. But I had read a science-fiction novel a few years ago where it was true, and my brain had, based on some feeling of "surprising but true fact," accidentally updated into thinking it was real. 😬

^{^}

There's still a divide between what different political factions read, of course, but I think it's rare for the explicit political orientation of the author to be the dominant factor, rather than simply having different tastes, such as highbrow vs lowbrow stories. Even famous counterexamples like JK Rowling, Orson Scott Card, and Stephen King are still enjoyed by the majority of political opponents, even if there's controversy.

^{^}

Here are some more facts, because I love to debunk myths from fiction:

Knocking someone out, whether through chloroform or concussion, for any amount of time is dangerous and potentially lethal.
Cars (and gas cans in general) almost never explode, though they can certainly ignite.
Falling a short distance onto water can easily hurt or kill someone. How they hit the water is often more important than height.
Running away from a gunman is an effective defense if there's any significant distance from the shooter. Hitting a moving target is hard!
And of course, basically nothing on TV or in movies involving spaceships or laser guns is anywhere close to accurate.

^{^}

Casual oversimplification stands in contrast to stylistic oversimplification. It's fine to have plucky rebels fighting the greedy empire, as long as it's clear to the audience that the story is deliberately flattening the social conflict.

^{^}

In my experience, one of the more useful heuristics towards guiding an audience towards the truth is to highlight your uncertainty, as an author. Highlighting, here, means calling the reader's attention to something and encouraging them to think about it for themselves before you reveal how the story handles things. A skilled author, I claim, can encourage truthseeking, even if they happen to be wrong on the object-level.

^{^}

LESSWRONG
LW

LESSWRONG
LW

52

The Unreasonable Effectiveness of Fiction

52

52

AI in Particular

What's Going On Here?

Authorial Responsibility