A formalisation of the ideas in this sequence in higher-order logic, including machine verified proofs of all the theorems, is available here.
"subagent [C] that could choose U" -- do you mean U⊆Ctrl(C) or U⊆Ensure(C) or neither of these? Since Ctrl is not closed under unions, I don't think the controllables version of "could choose" is closed under coarsening the partition. (I can prove that the ensurables version is closed; but it would have been nice if the controllables version worked.)
ETA: Actually controllables do work out if I ignore the degenerate case of a singleton partition of the world. This is because, when considering partitions of the world, ensurables and controllables are almost the same thing.
because whereas math-proof data-centers might result in our inadvertent death, something that refers to what humans want might result in deliberate torture.
I want to note that either case of screwing up this badly currently feels pretty implausible to me.
I have something suggestive of a negative result in this direction:
Let C be the prime-detector situation from Section 2.1 of the coarse worlds post, and let p:W→W be the (non-surjective) function that "heats" the outcome (changes any "C" to an "H"). The frame p∘(C) is clearly in some sense equivalent to the one from the example (which deletes the temperature from the outcome) -- I am using my version just to stay within the same category when comparing frames. As a reminder, primality is not observable in C but is observable in p∘(C).Claim: No frame of the form ExternalV(C) is biextensionally equivalent to p∘(C)Proof Idea: Image(ExternalV(C))=Image(C)≠Image(p∘(C))The kind of additional observability we get from coarsening the world seems in this case to be very different from the kind that comes from externalising part of the agent's decision.
With the other problem resolved, I can confirm that adding an A=∅ escape clause to the multiplicative definitions works out.
Using the idea we talked about offline, I was able to fix the proof - thanks Rohin!Summary of the fix:When D1 and D2 are defined, additionally assume they are biextensional (take their biextensional collapse), which is fine since we are trying to prove a biextensional equivalence. (By the way this is why we can't take b1=b2, since we might have A⊇B1≠B2⊆A after biextensional collapse.) Then to prove h=hf1, observe that for all b∈B1, b∙1h(b′2)=b∙1h(b2) which means b⋆1h(b′2)=b⋆1f1, hence h(b′2)=f1 since a biextensional frame has no duplicate columns.
I presume the fix here will be to add an explicit A=∅ escape clause to the multiplicative definitions. I haven't been able to confirm this works out yet (trying to work around this), but it at least removes the null counterexample.
How is this supposed to work (focusing on the h=hf1 claim specifically)?
Earlier, hf1 was defined as follows:
given by gf1(b1)=b1⋅1f1 and hf1(b2)=f1
but there is no reason to suppose f1=r above.
It suffices to establish that Ensure(CTi)⊇Ensure(CTj)
I think the Ti and Tj here are supposed to be V and U