I think it's not a fluke at all. Decision theory gave us a formal-seeming way of thinking about the behaviour of artificial agents long in advance of having anything like them, you have to believe you can do math about AI in order to think that it's possible to arrest the problem before it arrives, and also drawing this analogy between AI and idealised decision theory agents smuggles in a sorcerers apprentice frame (where the automaton arrives already strong, and follows instructions in an explosively energetic and literal way) that makes AI seem inherently dangerous.

So to be the most strident and compelling advocate of AI safety you had to be into decision theory. Eliezer exists in every timeline.

The usual thought, I guess. We could build forums that're sufficiently flexible that they could have features like this added to them without any involvement from hosts (in this case I'd implement it as a Proposal/mass commitment to read 'post of the day's, and the introduction of a 'post of the day' tag. I don't think this even requires radical extensibility, just the tasteweb model), and we should build those instead of building more single purpose systems that are even less flexible than the few-purpose systems we already had.

Do we know whether wolves really treat scent marks as boundary markers.

Some confusing things about wolf territoriality is they frequently honestly signal their locations through howling while trying (and imo failing?) to obfuscate their number in the way that they howl.

Not a coincidence, there are practical reasons borders end up on thresholds. A sort of quantization that happens in the relative strength calculation. Two models:

• Simple model: You could say the true border can be defined in terms of the amount of time or effort it takes to get there from the cat's houses. It takes a certain amount of time and effort to get over a fence, so if the true border is (from tuxedo's house) between distance tud + yd and tud + yd + fd, then the border in practice will end up being exactly on the fence, because you can't put a border halfway up the fence, or the situation would look the same if you did.
• A more accurate model: The border is measured in terms of how hard it is to defend a space from being used by the other side. I'd guess that cats become vulnerable to attack when they mount a fence (same as humans crossing a river), either coming or going, so extending your territory beyond the fence is difficult. If your strength is higher than the amount of strength it takes to defend everything before the threshold, but lower than the amount it takes to cross the threshold and then defend some on the other side, then the border will be exactly on the threshold.

I'm pretty sure I'd predict no for 1. Cats don't seem to care about that stuff.

For 2, I'm not sure, if there were a hole in the fence, I'd expect confrontations to happen there because that's a chokepoint where a cat could get through safely if the other one wasn't standing on the other side, and maybe the chokepoint is a vulnerability threshold, too. Chokepoints are thresholds for projectile combat (because when you come through the defender sees you immediately but you don't spot them until they start shooting), cats may be partly characterizable as stealth projectiles.

Also worth noting is that, eg, dogs, they engage in "boundary aggression" at things like fences, but experiments show that they're doing it for the love of the game. If you remove the fence, hostilities cease. Cats may have some of this going on as well. They may on some level enjoy yelling and acting tough while being in no risk of having to actually fight.

3: Yeah, but because it makes the relative strength calculation harder. A fence is a blessed device that allows cats to get a good look at each other without engaging. I wish humans had something like that. (A hole in a fence may also be a good device for this)

A schelling point is an arbitrary default choice converged upon without communication when agreement is needed more than correctness. A territorial border between animals is an extremely non-arbitrary result of often very thorough tests of relative strength and communications of will. Animal borders are opposite to schelling points.

Borders between human territories are pretty arbitrary, we don't really have the kind of bounded conflict that can produce a relative strength estimate any more (some of us used to), and most of us engage in antinductive commitment races by propagandising mythic histories about the legitimacy of our land claims (I don't believe in that shit though, personally). The present order truly seems to be satisfied with schelling points for borders, it doesn't matter what you choose, as long as we can agree, and never disagree, and whatever we agree about is the true border.

But animal borders aren't arbitrary, they're constantly renegotiated. The negotiations may be partly tacit, but there's nothing whimsical or symbolic about the outcomes. The animals know where the resources are, they know how much they want them, they know their neighbors, they know how often the neighbors come to check their border so they can estimate the amount of pressure they face, and they know the risks of getting into a fight with their neighbor, so they're able to make really pretty rational calculations to decide where the borders are.

I don't know what you're asking. The answer is either trivial or mu depending on what you mean by specific form. I think if you could articulate what you're asking you wouldn't have to ask it.

The VNM axiom isn't about road trips, a utility function is allowed to value different things at different times because the time component distinguishes those things. You aren't addressing VNM utility here. You're writing about a misunderstanding of it that you had.
You die if you have VNM cycles. A superior trader eats you (People feel like they can simply stop communicating with the sharps and retire to a simple life in the hills, but this is a very costly solution and I'd prefer to find a real one). You stop existing. This is kind of a much more essential category of instrumental vice than like "I don't equate money to utility" type stuff (which I wouldn't call a vice).

One criticism of decision theory that you could explore is that many practical philosophy enjoyers would find it difficult write utility functions that compose scripted components (like "I want A, then B, then C, then A") with nonscripted components ("I will always instantly trade X for Y, and Y for Z"), that we may need higher level abstractions on top of the basics to help people to stop conflating ABC with XYZ... but... is it really going to be complicated? That one doesn't seem like it's going to be complicated to me.

What does seem difficult is expressing constrained indifference about utility function changes. Something that seems to be common in humans (eg, I'm indifferent to the change/annihilation of my values if it's being done by beautiful and cool things like love, literary fiction, or reason, but I hate it if it's being done by ugly or stupid or hostile things.) and is needed for ASI alignment (corrigibility), but it seems tricky to define a utility function that permits it. (though again I don't know whether it turns out to be tricky in practice)

A utility function that enjoys moving between those places isn't the same as a utility function with cycles, which would trade unlimited time money for tickets to them that it never cashes.

The argument against this is that is also going to be somewhat instrumental in flavour but more along the lines of like, that's a known attracter that few who matter want to be in.

In a world that has ASI, a much better way of maintaining the integrity of the audit system by building it to be intelligent enough to tell whether it's being fooled, and with a desire of its own to stay neutral. Which I guess is like being multistakeholder, since you both will have signed off on its design.

But in such a world, the audit system would be a feature of the brain of the local authorities. You would co-design yourselves in such a way that you have the ability to make binding promises (or if you're precious about your design, co-design your factories in such a way that they have the ability to verify that your design can make binding promises (or co-design your factory factories to ...)). This makes you a better/viable at all trading partner. You have the option of not using it except when it benefits you. But having it means that they can simply ask you whether your galaxy contains any optimal 17 square packings, and you send them an attestation that no when you need to pack 17 squares you're using the socially acceptable symmetrical, suboptimal packings, and if it has a certain signature then they know you weren't capable of faking this message.

You really don't want to lack this ability.

But starting from the outset with little shared resources and (obviously) allowing establishment of shared projects using their resources by agreement between the stakeholders doesn't seem much different from some de novo process of establishing such shared projects with no direct involvement of individuals

You're speaking as if we're starting with strict borders and considering renegotiating, for most of the resources in the universe and also on the planet this is not the case, ownership of space is a taboo, ownership over ocean resources is shared, at least on the nation level. It's as if humans have shame, sense the absurdity of it all, and on some level fear enclosed futures. I think shared ownership (which is not really ownership) is a more likely default, shared at least between more than one person, if not a population.

But to the point, I don't think we know that the two starting points lead to equivalent outcomes. My thesis is generally that it's very likely that transparency (then coordination) physically wins out under basically any natural starting conditions, but even if the possibility that some coordination problems are permanent is very small, I'd prefer if we avoided the risk. But I also notice that there may be some governance outcomes that make shared start much less feasible than walled start.