I've been running into something I think of as "The Narrow Band Dilemma" where a moderate ethical position is fragile because it is in a battle on two fronts between a popular more pure ethical position and a popular unethical or a-moral position.

The first example is ethically produced meat / free-range farming, where a slightly more expensive product tries to find a market that loses patrons on both sides, either people who don't care about where their meat comes from, or care more about the expense than the ethics on the one hand, and the people who forgo meat altogether (vegans / vegetarians) on the other. As messaging successfully advertises the benefits of ethical meat production, it gains market share from one end of the spectrum (the "I care about ethics but can't justify the additional expense") but loses them on the other because the heightened awareness of animal cruelty drives ethical meat eaters away from eating meat altogether.

I noticed another example today in moderate Christianity, where it is flanked by fundamentalism and atheism. The more that Christians want to align their beliefs with modern secular ethics (moving away from fundamentalism), the more they are likely to leave the faith altogether.

Now, I'm not actually saying this is a problem, just a phenomenon I've noticed, I'm an atheist, so don't mind if more people come to think as I do, but I'd also like more Christians to be more moderate, but I can see it's difficult to build critical mass when they exist in a narrow band. I am in the narrow band when it comes to ethical meat, and I have seen how long it has taken for ethical meat products to become ubiquitous and accessible, and truly ethical real meat (lab grown) is still a way off. I imagine, if a much larger market had emerged (without the pressures of the narrow band), it's possible this could be available already.

You're right, but the term teme covers much more than that, for instance it's also relevant to the development of AI agents, and AI self-editing / self-improvement. Although, identifying these systems as virus-like (because of the replication mechanism) might be instructive (as a red-flag).

I agree, it's not coming across at all well at present, needs a rewrite, give me a couple of weeks :)

I take your point, I think it needs a rewrite, I have not been nearly clear enough, and your notes are helpful in pointing me to areas I need to clarify. I have replies to your points here, but I should get my ducks-in-a-row before making them, so I don't end up contradicting myself. Thanks for your comment.

Thanks Adele,

I appreciate your comment, and will take some time to process it and read the links. This is definitely not an area I have any expertise in and I'm not meaning to propose that this is how gravity actually works in reality—it's more an illustration that something gravity-like, and elements that are like atoms or systems etc can arise out of very simple and random rules without the need for fine-tuning, and that constants (or regularities) can be arrived at by means of natural equilibria rather than being lucked upon, or designed.

But I probably haven't made this clear. It was something I actually wrote I while ago and have only recently published here, so it may require a re-write, clarifying my intention and incorporating the points you've raised. You're the first to provide a rigorous rebuttal for it so far, so I appreciate you lending your expertise in this respect.

Ah, yes now you've jogged my memory about all the attempted expansionism in between. You make a solid case that they didn't step outside of the expand-or-die dichotomy willingly.

The point I'm trying to make is that the third option was there (perhaps it wasn't feasible before WWII, I'm not sure), but the third option (mutually beneficial trade and cooperation) ended up sustaining Japan from WWII to the present without the need for expansion.

The point of the post is that often there is often a third option outside of expand-or-die, and it's worth questioning what that could be in any given problem. But thanks for all the very good points—I absolutely agree with you that there have been civilisations that have had to, or have seemed to have to, expand in order to survive. Thanks for your well-considered points, and the spot-on history (apologies for the patchiness of mine).

I take your point, it requires everyone to behave themselves (I'm actually familiar with this history, I went down a Japanese history rabbit hole about this time last year, fascinating), but if we continue with Japan we find that due to a third option of trading-with-other-nations (beginning with the Meiji restoration I think...) Japan continues to operate as a sovereign nation without the need to expand (with the exception of its ill-fated and frankly bonkers attempt to expand in WWII...).

So, again it's good to look for answers outside the paradigm of expand or die. Cooperation and trade are non-zero-sum options that are available in the messy and therefore less theoretically-bound real world, as opposed to a formal game theory scenario.

But I think the expansionist trap you describe is a real thing and an important cautionary tale, which could perhaps be applied to our modern perpetual growth model of economics and its attendant consumerism (here I am sounding like a first year sociology major).

True, though there are many examples of conquerors who expanded for the sake of an expansionist philosophy or glory: Alexander the Great, The Mongols, The Assyrians, The Crusades... off the top of my head. The Germans in WWII definitely justified expansion for the sake of living space (Lebensraum), so there are examples of expansion at least being justified in the way you mention. And of course colonialism is justified in the same way.

I think what you're saying is logical, but the example, being metaphorical, is more to illustrate that we should question critically what it is we actually want before conceding a price to pay for it. As you say, it might be necessary, but it also might not.

Humans profess to care about everyone a lot more than they really do, because doing that (and even thinking that) is strategically useful.

A bit bleak... but yes, your logic checks out, and hence why coordination problems are so sticky (I did sort of claim to solve the problem didn't I? Oops, back to the drawing board).

I love this silly side of Yudkowsky.