Interesting. Thanks. How did you arrive at the above picture? Any sources of information you'd recommend in particular?

After reading about Trump's actions w.r.t. Greenland, I'm updating further away from

• Trump's policies/actions are at least partially aimed at pursuing the national interests of the US (albeit possibly in a very misguided/incompetent way)

and further in favor of both

• Trump has gone insane,
• and/or Trump is intentionally acting on behalf of Putin.

I'd like to find more/better sources of evidence about "what is the US executive branch optimizing for?"; curious to hear suggestions.

(Also, to Americans: How high/low salience is the issue in the US? Also: curious to read your analysis of your chief executive's behavior.)

Dominic Cummings (former Chief Adviser to the UK PM) has written some things about nuclear strategy and how it's implemented in practice. IIUC, he's critical of (i.a.) how Schelling et al.'s game-theoretic models are (often naively/blindly) applied to the real world.

I updated a bit towards thinking that incompetence-at-reasoning is a more common/influential factor than I previously thought. Thanks.

However: Where do you think that moral realism comes from? Why is it a "thorny" issue?

social-media-like interfaces for uncovering group wisdom and will at larger scales while eliciting more productive discourse

That seems like it might significantly help with raising the sanity waterline, and thus help with coordinating on AI x-risk, and thus be extremely high-EV (if it's successfully implemented, widely adopted, and humanity survives for a decade or two beyond widespread adoption). ^[1]

Do you think it would be practically possible with current LLMs to implement a version of social media that promoted/suggested content based on criteria like

• validity of reasoning,
• concrete factual claims (whether correct or incorrect), vs vibing,
• thoughtful long-form writing, vs memeing,
• good-faith debate, vs e.g. adversarial tribalistic signaling?

1. The "widely adopted" part seems difficult to achieve, though. The hypermajority of genpop humans would probably just keep scrolling TikTok and consuming outrage porn on X, even if Civilization 2.0 Wholesome Social Media were available. ↩︎

This discourse structure associates related claims and evidence, [...]

To make it practically possible for non-experts to efficiently make sense of large, spread-out collections of data (e.g. to answer some question about the discourse on some given topic), it's probably necessary to not only rapidly summarize all that data, but also translate it into some easily-human-comprehensible form.

I wonder if it's practically possible to have LMs read a bunch of data (from papers to Twitter "discourse") on a given topic, and rapidly/on-demand produce various kinds of concise, visual, possibly interactive summaries of that topic? E.g. something like this, or a probabilistic graphical model, or some kind of data visualization (depending on what aspect of what kind of topic is in question)?

Ideally perhaps, raw observations are reliably recorded, [...]

Do you have ideas for how to deal with counterfeit observations or (meta)data (e.g. deepfaked videos)?

Given that the basic case for x-risks is so simple/obvious ^[[1]] , I think most people arguing against any risk are probably doing so due to some kind of myopic/irrational subconscious motive. (It's entirely reasonable to disagree on probabilities, or what policies would be best, etc.; but "there is practically zero risk" is just absurd.)

So I'm guessing that the deeper problem/bottleneck here is people's (emotional) unwillingness to believe in x-risks. So long as they have some strong (often subconscious) motive to disbelieve x-risks, any conversation about x-risks is liable to keep getting derailed or be otherwise very unproductive. ^[[2]]

I think some common underlying reasons for such motivated disbelief include

1. Subconscious Map-Territory confusions:
   1. Believing X makes X feel more real. And so, System 1 "solves" x-risks by disbelieving them away.
   2. System 1 makes decisions -- including decisions for what to believe -- under the assumption that the current Map is the Territory. For example: If the current Map says that "x-risks are not real and that's good because now I can keep making money developing AGI capabilities", then according to that Map, it would be bad to update towards believing that x-risks are real (because then maybe you'd needlessly stop making money!)
2. Cognitive immune response. Lots of humans have a strong subconscious instinct along the lines of "If a clever human is trying, with abstract arguments, to convince me of X, and I anticipate that believing X would change my policy/behavior in ways that feel unpleasant, then I reject/disbelieve X". ^[[3]]

I'm not sure what the best approaches are to addressing the above kinds of dynamics. Trying to directly point them out seems likely to end badly (at least with most neurotypical people). If you can somehow get people to (earnestly) do them, small mental exercises like Split and Commit or giving oneself a line of retreat might help for (1.)? For (2.), maybe

• avoid trying to persuade; instead ask questions that prompt the person to think concretely about the situation themself,
• show them something tangible, like some AI model doing something impressive, or that slowed-down video of a train station (along with "this is what humans would look like to an entity that thinks 100 times faster -- statues")?

If you try the above, I'd be curious to see a writeup of the results.

1. Building a species of superhumanly smart & fast machine aliens without understanding how they work seems very dangerous. And yet, various companies and nations are currently pouring trillions of dollars into making that happen, and appear to be making rapid progress. (Experts disagree on whether there's a 99% chance we all die, or if there's only a 10% chance we all die and a 90% chance some corporate leaders become uncontested god-emperors, or if we end up as pets to incomprehensible machine gods, or if the world will be transformed beyond human comprehension and everyone will rely on personal AI assistants to survive. Sounds good, right?) ↩︎

2. A bit like trying to convince a deeply religious person via rational debate. It's not really about the evidence/reasoning. ↩︎

3. I wouldn't be too surprised if this kind of instinct were evolved, rather than just learned. Even neurotypical humans try to hack each other all the time, and clever psychopaths have probably been around for many, many generations. ↩︎

Think of the stuff that, when you imagine it, feels really yummy.

Also worth taking into consideration: things that feel anti-yummy. Fear/disgust/hate/etc are also signals about your values.