Related to: Commonsense Good, Creative Good (and my comment); Ethical Injunctions.

Epistemic status: I’m fairly sure “ethics” does useful work in building human structures that work. My current explanations of how are wordy; I think there should be a briefer way to conceptualize it; I hope you guys help me with that.

Introduction

It is intractable to write large, good software applications via spaghetti code – but it’s comparatively tractable using design patterns (plus coding style, attention to good/bad codesmell, etc.).

I’ll argue it is similarly intractable to have predictably positive effects on large-scale human stuff if you try it via straight consequentialism – but it is comparatively tractable if you use ethical heuristics, which I’ll call “ethical design patterns,” to create situations that are easier to reason about. Many of these heuristics are honed by long tradition (eg “tell the truth”; “be kind”), but sometimes people successfully craft new “ethical design patterns” fitted to a new circumstance, such as “don’t drink and drive.”

I will close with a discussion of what I think it would take to craft solid ethical heuristics near AI development, and to be able to thereby put energy into AI safety efforts in a way that is less in a fight with others’ desires to avoid totalitarianism, economic downturns, or other bad things.

Intuitions and ground truth in math, physics, coding

We humans navigate mostly by intuition.^[1] We can do this, not because our intuitions match up with the world from birth (they mostly don’t), but because:

We check our intuitions against data, and revise those that make wrong predictions; and
We design our built world to be intuitively navigable.

I’ll spell this out a bit more in the case of math, physics, and coding. Then, with this example in hand, I'll head to ethics.

We revise our intuitions to match the world. Via deliberate work.

Humans predict math and physics using informal/intuitive guesswork (mostly) and observation and formal proof (occasionally, but with higher credibility when we do check).

How do we get good-enough intuitions to make this work? Partly, we start with them: humans have some intuition-reality match from the get-go. Partly, we revise them fairly automatically with experience, as we play around and "get a feel for things."

But also, partly, we revise them on purpose, with full verbal consciousness: great phys...

Missing Moods