Previous proof post is here.

Theorem 2: Isomorphism Theorem: For (causal, pseudocausal, acausal, surcausal) $Θ^{s t}$ or $Θ^{ω}$ which fulfill finitary or infinitary analogues of all the defining conditions, $↑ (Θ^{s t})$ and $↓ (Θ^{ω})$ are (causal, pseudocausal, acausal, surcausal) hypotheses. Also, $↑$ and $\to^{s t}$ define an isomorphism between $Θ$ and $Θ^{s t}$ , and $↓$ and $\to^{ω}$ define an isomorphism between $Θ$ and $Θ^{ω}$ .

Proof sketch: The reason this proof is so horrendously long is that we've got almost a dozen conditions to verify, and some of them are quite nontrivial to show and will require sub-proof-sketches of their own! Our first order of business is verifying all the conditions for a full belief function for $↑ (Θ^{s t})$ . Then, we have to do it all over again for $↓ (Θ^{ω})$ . That comprises the bulk of the proof. Then, we have to show that taking a full belief function $Θ$ and restricting it to the infinite/finite levels fulfills the infinite/finite analogues of all the defining conditions for a belief function on policies or policy-stubs, which isn't quite as bad. Once we're done with all the legwork showing we can derive all the conditions from each other, showing the actual isomorphism is pretty immediate from the Consistency condition of a belief function.

Part 1:Let's consider $↑ (Θ^{π_{s t}})$ . This is defined as: $↑ (Θ^{s t}) (π_{p a}) := ⋂_{π_{s t} \leq π_{p a}} (p r_{*}^{π_{p a}, π_{s t}})^{- 1} (Θ^{s t} (π_{s t}))$

We'll show that all 9+2 defining conditions for a belief function are fulfilled for $↑ (Θ^{s t})$ . The analogue of the 9+2 conditions for a $Θ_{s t}$ is:

1: Stub Nirvana-free Nonemptiness: $\forall π_{s t} : Θ^{s t} (π_{s t}) \cap N F \neq \emptyset$

2: Stub Closure: $\forall π_{s t} : Θ^{s t} (π_{s t}) = ¯ ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯ ¯ Θ^{s t} (π_{s t})$

3: Stub Convexity. $\forall π_{s t} : Θ^{s t} (π_{s t}) = c . h (Θ^{s t} (π_{s t}))$

4: Stub Nirvana-free Upper-Completion.

$\forall π_{s t} : Θ^{s t} (π_{s t}) \cap N F = ((Θ^{s t} (π_{s t}) \cap N F) + M^{s a} (F^{N F} (π_{s t}))) \cap M^{a} (F (π_{s t}))$

5: Stub Restricted Minimals:

$\exists λ^{⊙}, b^{⊙} \forall π_{s t} : (λ μ, b) \in {(Θ^{s t} (π_{s t}))}_{s t}^{min} \to λ + b \leq λ^{⊙} + b^{⊙}$

6: Stub Normalization: ${inf}_{π_{s t}} E_{Θ^{s t} (π_{s t})} (0) = 0$ and ${sup}_{π_{s t}} E_{Θ^{s t} (π_{s t})} (1) = 1$

7: Weak Consistency: $\forall π_{s t}^{l o}, π_{s t}^{h i} \geq π_{s t}^{l o} : p r_{*}^{π_{s t}^{h i}, π_{s t}^{l o}} (Θ^{s t} (π_{s t}^{h i})) \subseteq Θ^{s t} (π_{s t}^{l o})$

8: Stub Extreme Point Condition: for all $M, π_{s t}^{l o}$ :

$M \in (Θ^{s t} (π_{s t}^{l o}))^{x m i n} \cap N F \to$

$\exists π \geq π_{s t}^{l o} \forall π_{s t}^{h i} : (π > π_{s t}^{h i} \geq π_{s t}^{l o} \to (\exists M^{'} \in Θ^{s t} (π_{s t}^{h i}) \cap N F : p r_{*}^{π_{s t}^{h i}, π_{s t}^{l o}} (M^{'}) = M))$

9: Stub Uniform Continuity: The function $π_{s t} \mapsto (p r_{*}^{\infty, π_{s t}})^{- 1} (Θ^{s t} (π_{s t}) \cap N F \cap {\leq ⊙})$ is uniformly continuous.

C: Stub Causality: $\forall π_{s t}, M \in Θ^{s t} (π_{s t}) \exists o f \forall π_{s t}^{'} : o f (π_{s t}) = M \land o f (π_{s t}^{'}) \in Θ^{s t} (π_{s t}^{'})$

where the outcome function $o f$ is defined over all stubs.

P: Stub Pseudocausality:

$\forall π_{s t}, π_{s t}^{'} : ((M \in Θ^{s t} (π_{s t}) \land supp (M) \subseteq F^{N F} (π_{s t}^{'})) \to M \in Θ^{s t} (π_{s t}^{'}))$

Let's begin showing the conditions. But first, note that since we have weak consistency, we can invoke Lemma 6 to reexpress $↑ (Θ^{s t}) (π_{p a})$ as $⋂_{n} (p r_{*}^{π_{p a}, π_{p a}^{n}})^{- 1} (Θ^{s t} (π_{p a}^{n}))$ Where $π_{p a}^{n}$ is the n'th member of the fundamental sequence of $π_{p a}$ .

Also note that, for all stubs, $↑ (Θ^{s t} (π_{s t})) = Θ^{s t} (π_{s t})$ . We'll be casually invoking this all over the place and won't mention it further.

Proof: By Lemma 6 with weak consistency, $↑ (Θ^{s t} (π_{s t})) = ⋂_{n \geq m} (p r_{*}^{π_{s t}, π_{s t}^{n}})^{- 1} (Θ^{s t} (π_{s t}^{n}))$ Now, m can be anything we like, as long as it's finite. Set m to be larger than the maximum timestep that the stub is defined for. Then $π_{s t}^{n} = π_{s t}$ no matter what n is (since it's above m) and projection from a stub to itself is identity, so the preimage is exactly our original set $Θ^{s t} (π_{s t})$ .

We'll also be using another quick result. For all stubs $π_{s t}^{h i} \geq π_{s t}^{l o}$ , given stub causality,

$p r_{*}^{π_{s t}^{h i}, π_{s t}^{l o}} (Θ^{s t} (π_{s t}^{h i})) = Θ^{s t} (π_{s t}^{l o})$

Proof: Fix an arbitrary point $M \in Θ^{s t} (π_{s t}^{l o})$ . By causality, we get an outcome function which includes $M$ , giving us that there's something in $Θ^{s t} (π_{s t}^{h i})$ that projects down onto $M$ . Use weak consistency to get the other subset direction.

Condition 1: Nirvana-free Nonemptiness.

Invoking Stub Nirvana-free Nonemptiness, $\forall π_{s t} : Θ^{s t} (π_{s t}) \cap N F \neq \emptyset$ so we get nirvana-free nonemptiness for $↑ (Θ^{s t}) (π_{s t})$ .

Now, assume $π_{p a}$ is not a stub. By stub-bounded-minimals, there is some $λ^{⊙} + b^{⊙}$ bound on the set of minimal points, regardless of stub. Let $(Θ^{s t} (π_{p a}^{n}))^{c l i p}$ be $Θ^{s t} (π_{p a}^{n}) \cap {\leq ⊙} \cap N F$

This contains all the minimal nirvana-free points for $π_{p a}^{n}$ . This set is nonempty because we have stub nirvana-free nonemptiness, so a nirvana-free point $M$ exists. We have stub-closure and stub minimal-boundedness, so we can step down to a minimal nirvana-free point below $M$ , and it obeys the $λ^{⊙} + b^{⊙}$ bound.

Further, by weak consistency and projection preserving $λ$ and $b$ and nirvana-freeness,

$p r_{*}^{π_{p a}^{n + 1}, π_{p a}^{n}} ((Θ^{s t} (π_{p a}^{n + 1}))^{c l i p}) \subseteq (Θ^{s t} (π_{p a}^{n}))^{c l i p}$

Invoking Lemma 9, the intersection of preimages of these is nonempty. It's also nirvana-free, because if there's nirvana somewhere, it occurs after finite time, so projecting down to some sufficiently large finite stage preserves the presence of Nirvana, but then we'd have a nirvana-containing point in a nirvana-free set $(Θ^{s t} (π_{p a}^{n}))^{c l i p}$ , which is impossible. This is also a subset of the typical intersection of preimages used to define $↑ (Θ^{s t}) (π_{p a})$ . Pick an arbitrary point in said intersection of preimages of clipped subsets.

Bam, we found a nirvana-free point in $↑ (Θ^{s t}) (π_{p a})$ and we're done.

Time for conditions 2 and 3, Closure and Convexity. These are easy.

$↑ (Θ^{s t}) (π_{p a}) = ⋂_{n} (p r_{*}^{π_{p a}, π_{p a}^{n}})^{- 1} (Θ^{s t} (π_{p a}^{n}))$

The preimage of a closed set (stub-closure) is a closed set, and the intersection of closed sets is closed, so we have closure.

Also, $p r_{*}^{π_{p a}, π_{s t}^{n}}$ is linear, so the preimage of a convex set (stub-convexity) is convex, and we intersect a bunch of convex sets so it's convex as well.

Condition 4: Nirvana-free upper completion.

Let $M \in↑ (Θ^{s t}) (π_{p a}) \cap N F$ . Let's check whether $M + M^{*}$ (assuming that's an a-measure and $M^{*}$ is nirvana-free) also lies in the set. A sufficient condition on this given how we defined things is that for all $π_{p a}^{n}$ , $p r_{*}^{π_{p a}, π_{p a}^{n}} (M + M^{*}) \in Θ^{s t} (π_{p a}^{n})$ , as that would certify that $M + M^{*}$ is in all the preimages.

$p r_{*}^{π_{p a}, π_{p a}^{n}}$ is linear, so $p r_{*}^{π_{p a}, π_{p a}^{n}} (M + M^{*}) = p r_{*}^{π_{p a}, π_{p a}^{n}} (M) + p r_{*}^{π_{p a}, π_{p a}^{n}} (M^{*})$

The first component is in $Θ^{s t} (π_{s t})$ , obviously. And then, by stub nirvana-free-upper-completion, we have a nirvana-free a-measure plus a nirvana-free sa-measure (projection preserves nirvana-freeness), making a nirvana-free a-measure (projection preserves a-measures), so $p r_{*}^{π_{p a}, π_{p a}^{n}} (M) + p r_{*}^{π_{p a}, π_{p a}^{n}} (M^{*})$ is in $Θ^{s t} (π_{p a}^{n}) \cap N F$ , and we're done.

Condition 5: Bounded-Minimals

So, there is a critical $λ^{⊙} + b^{⊙}$ value by restricted-minimals for $Θ^{s t}$

Fix a $π_{p a}$ , and assume that there is a minimal point $M$ in $↑ (Θ^{s t}) (π_{p a})$ with a $λ + b$ value that exceeds the bound. Project $M$ down into each $Θ^{s t} (π_{p a}^{n})$ . Projection preserves $λ$ and $b$ so each of these projected points $M_{n}$ lie above some $M_{n}^{min}$ .

Now, invoke Lemma 7 to construct a $M_{n}^{'} \in M^{a} (F (π_{p a}))$ (or the nirvana-free variant) that lies below $M$ , and projects down to $M_{n}^{min}$ . Repeat this for all n. All these $M_{n}^{'}$ points are a-measures and have the standard $λ^{⊙} + b^{⊙}$ bound so they all lie in a compact set and we can extract a convergent subsequence, that converges to $M^{'}$ , which still obeys the $λ^{⊙} + b^{⊙}$ bound.

$M^{'}$ is below $M$ because $({M} - M^{s a} (F (π_{p a}))) \cap M^{a} (F (π_{p a}))$ (or the nirvana-free variant) is a closed set. Further, by Lemma 10, $M^{'}$ is in the defining sequence of intersections for $↑ (Θ^{s t}) (π_{p a})$ . This witnesses that $M$ isn't minimal, because we found a point below it that actually obeys the bounds. Thus, we can conclude minimal-point-boundedness for $↑ (Θ^{s t})$ .

Condition 6: Normalization. We'll have to go out of order here, this can't be shown at our current stage. We're going to have to address Hausdorff continuity first, then consistency, and solve normalization at the very end. Let's put that off until later, and just get extreme points.

Condition 8: Extreme point condition:

The argument for this one isn't super-complicated, but the definitions are, so let's recap what condition we have and what condition we're trying to get.

Condition we have: for all $M, π_{s t}^{l o}$ :

$M \in (Θ^{s t} (π_{s t}^{l o}))^{x m i n} \cap N F \to$

Condition we want: for all $M, π_{s t}$ ,

$M \in (Θ^{s t} (π_{s t}))^{x m i n} \cap N F \to \exists π > π_{s t}, M^{'} : M^{'} \in↑ (Θ^{s t}) (π) \cap N F \land p r_{*}^{π, π_{s t}} (M^{'}) = M$

Ok, so $M \in (Θ^{s t} (π_{s t}))^{x m i n} \cap N F$ By the stub extreme point condition, there's a $π \geq π_{s t}$ , where, for all $π_{s t}^{h i}$ that fulfill $π > π_{s t}^{h i} \geq π_{s t}$ , there's a $M^{'} \in Θ^{s t} (π_{s t}^{h i}) \cap N F$ , where $p r_{*}^{π_{s t}^{h i}, π_{s t}} (M^{'}) = M$ .

Lock in the $π$ we have. We must somehow go from this to a $M^{'} \in Θ (π) \cap N F$ that projects down to our point of interest. To begin with, let $π^{n}$ be the n'th member of the fundamental sequence for $π$ . Past a certain point m, these start being greater than $π_{s t}$ . The $M^{'} \in Θ^{s t} (π^{n}) \cap N F$ which projects down to $M$ that we get by the stub-extreme-point condition will be called $M_{n}^{^{'} l o}$ . Pick some random-ass point in $(p r_{*}^{π, π_{s t}^{n}})^{- 1} (M_{n}^{^{'} l o})$ and call it $M_{n}^{^{'} h i}$ .

$M_{n}^{^{'} h i}$ all obey the $λ$ and $b$ values of $M$ , because it projects down to $M$ . We get a limit point of them, $M^{'}$ , and invoking Lemma 10, it's also in $↑ (Θ^{s t}) (π)$ . It also must be nirvana-free, because it's a limit of points that are nirvana-free for increasingly late times. It also projects down to $M$ because the sequence $M_{n}^{^{'} h i}$ was wandering around in the preimage of $M$ , which is closed.

Condition 9: Hausdorff Continuity:

Ok, this one is going to be fairly complicated. Remember, our original form is:

"The function $π_{s t} \mapsto (p r_{*}^{\infty, π_{s t}})^{- 1} (Θ^{s t} (π_{s t}) \cap N F \cap {\leq ⊙})$ is uniformly continuous"

And the form we want is:

"The function $π_{p a} \mapsto (p r_{*}^{\infty, π_{p a}})^{- 1} (↑ (Θ^{s t}) (π_{p a}) \cap N F \cap {\leq ⊙})$ is uniformly continuous"

Uniform continuity means that if we want an $ϵ$ Hausdorff-distance between two preimages, there's a $δ$ distance between partial policies that suffices to produce that. To that end, fix our $ϵ$ . We'll show that the $δ$ we get from uniform continuity on stubs suffices to tell us how close two partial policies must be.

So, we have an $ϵ$ . For uniform continuity, we need to find a $δ$ where, regardless of which two partial policies $π_{p a}$ and $π_{p a}^{'}$ we select, as long as they're $δ$ or less apart, the sets $(p r_{*}^{\infty, π_{p a}})^{- 1} (↑ (Θ^{s t}) (π_{p a}) \cap N F \cap {\leq ⊙})$ (and likewise for $π_{p a}^{'}$ ) are only $ϵ$ apart. So, every point in the first preimage must have a point in the second preimage only $ϵ$ distance away, and vice-versa. However, we can swap $π_{p a}$ and $π_{p a}^{'}$ (our argument will be order-agnostic) to establish the other direction, so all we really need to do is to show that every point in the preimage associated with $π_{p a}$ is within $ϵ$ of a point in the preimage associated with $π_{p a}^{'}$ .

First, $m := {log}_{γ} (δ)$ is the time at which two partial policies $δ$ apart may start differing. Conversely, any two partial policies which only disagree at-or-after m are $δ$ apart or less. Let $π_{s t}^{*}$ be the policy stub defined as follows: take the inf of $π_{p a}$ and $π_{p a}^{'}$ (the partial policy which is everything they agree on, which is going to perfectly mimic both of them up till time m), and clip things off at time m to make a stub. This is only $δ$ apart from $π_{p a}$ and $π_{p a}^{'}$ , because it perfectly mimics both of them up till time m, and then becomes undefined (so there's a difference at time m) Both $π_{p a}$ and $π_{p a}^{'}$ are $\geq π_{s t}^{*}$ .

Let $M$ be some totally arbitrary point in $(p r_{*}^{\infty, π_{p a}})^{- 1} (Θ^{s t} (π_{p a}) \cap N F \cap {\leq ⊙})$ . $M$ is also in $(p r_{*}^{\infty, π_{s t}^{*}})^{- 1} (Θ^{s t} (π_{s t}^{*}) \cap N F \cap {\leq ⊙})$ , because $M$ projects down to some point in $Θ^{s t} (π_{s t}^{*})$ that's nirvana-free.

Let $π_{p a}^{^{'} n}$ , where $n \geq m$ , be the n'th stub in the fundamental sequence for $π_{p a}^{'}$ . These form a chain starting at $π_{s t}^{*}$ and ascending up to $π_{p a}^{'}$ , and are all $δ$ distance from $π_{s t}^{*}$ .

Anyways, in $M^{a} (\infty)$ , we can make a closed ball $B_{\leq ϵ}$ of size $ϵ$ around $M$ . This restricts $λ$ and $b$ to a small range of values, so we can use the usual arguments to conclude that $B_{\leq ϵ}$ is compact.

Further, because $π_{s t}^{*}$ is $δ$ or less away from $π_{p a}^{^{'} n}$ , the two sets $(p r_{*}^{\infty, π_{s t}^{*}})^{- 1} (Θ^{s t} (π_{s t}^{*}) \cap N F \cap {\leq ⊙})$ and $(p r_{*}^{\infty, π_{p a}^{^{'} n}})^{- 1} (Θ^{s t} (π_{p a}^{^{'} n}) \cap N F \cap {\leq ⊙})$ are within $ϵ$ of each other, so there's some point of the latter set that lies within our closed $ϵ$ -ball.

Consider the set $⋂_{n \geq m} ((p r_{*}^{\infty, π_{p a}^{^{'} n}})^{- 1} (Θ^{s t} (π_{p a}^{^{'} n}) \cap N F \cap {\leq ⊙}) \cap B_{\leq ϵ})$

the inner intersection is an intersection of closed and compact sets, so it's compact. Thus, this is an intersection of an infinite family of nonempty compact sets. To check the finite intersection property, just observe that since preimages of the sets $↑ (Θ^{s t}) (π_{p a}^{^{'} n}) \cap N F \cap {\leq ⊙}$ get smaller and smaller as n increases due to weak-consistency but always exist.

Pick some arbitrary point $M^{'}$ from the intersection. it's $\leq ϵ$ away from $M$ since it's in the $ϵ$ -ball. However, we still have to show that $M^{'}$ is in $(p r_{*}^{\infty, π_{p a}^{'}})^{- 1} (↑ (Θ^{s t}) (π_{p a}^{'}) \cap N F \cap {\leq ⊙})$ to get Hausdorff-continuity to go through.

To begin with, since $M^{'}$ lies in our big intersection, we can project it down to any $Θ^{s t} (π_{p a}^{^{'} n}) \cap N F \cap {\leq ⊙}$ . Projecting it down to stage n makes $M_{n}^{'}$ . Let $M_{\infty}^{'}$ be the point in $↑ (Θ^{s t}) (π_{p a}^{'}) \cap N F \cap {\leq ⊙}$ defined by: $⋂_{n \geq m} (p r_{*}^{π_{p a}^{'}, π_{s t}^{n}})^{- 1} (M_{n}^{'})$

Well, we still have to show that this set is nonempty, contains only one point, and that it's in $↑ (Θ^{s t}) (π_{p a}^{'})$ , and is nirvana-free, to sensibly identify it with a single point.

Nonemptiness is easy, just invoke Lemma 9. It lies in the usual intersections that define $↑ (Θ^{s t}) (π_{p a}^{'})$ , so we're good there. If it had nirvana, it'd manifest at some finite point, but all finite projections are nirvana-free, so it's nirvana-free. If it had more than one point in it, they differ at some finite stage, so we can project to a finite $π_{p a}^{^{'} n}$ to get two different points, but they both project to $M_{n}^{'}$ , so this is impossible. Thus, $M_{\infty}^{'}$ is a legit point in the appropriate set. If the projection of $M^{'}$ didn't equal $M_{\infty}^{'}$ , then we'd get two different points, which differ at some finite stage, so we could project down to separate them, but they both project to $M_{n}^{'}$ for all n so this is impossible.

So, as a recap, we started with an arbitrary point $M$ in $(p r_{*}^{\infty, π_{p a}})^{- 1} (↑ (Θ^{s t}) (π_{p a})) \cap {\leq ⊙}$ , and got another point $M^{'}$ that's only $ϵ$ or less away and lies in $(p r_{*}^{\infty, π_{p a}^{'}})^{- 1} (↑ (Θ^{s t}) (π_{p a}^{'})) \cap {\leq ⊙}$ This argument also works if we flip $π_{p a}$ and $π_{p a}^{'}$ , so the two preimages are only $ϵ$ or less apart in Hausdorff-distance.

So, given some $ϵ$ , there's some $δ$ where any two partial policies which are only $δ$ apart have preimages only $ϵ$ apart from each other in Hausdorff-distance. And thus, we have uniform continuity for the function mapping $π_{p a}$ to the set of a-measures over infinite histories which project down to $↑ (Θ^{s t}) (π_{p a}) \cap N F \cap {\leq ⊙}$ Hausdorff-continuity is done.

Condition 7: Consistency.

Ok, we have two relevant things to check here. The first, very easy one, is that

$↑ (Θ^{s t}) (π_{p a}) = ⋂_{π_{s t} \leq π_{p a}} (p r_{*}^{π_{p a}, π_{s t}})^{- 1} (↑ (Θ^{s t}) (π_{s t}))$

From earlier, we know that $↑ (Θ^{s t}) (π_{s t}) = Θ^{s t} (π_{s t})$ , and from how $↑ (Θ^{s t}) (π_{p a})$ is defined, this is a tautology.

The other, much more difficult direction, is that

$↑ (Θ^{s t}) (π_{p a}) = ¯ ¯¯¯¯¯¯ ¯ c . h (⋃_{π \geq π_{p a}} p r_{*}^{π, π_{p a}} (↑ (Θ^{s t}) (π)))$

We'll split this into four stages. First, we'll show one subset direction holds in full generality. Second, we'll get the reverse subset direction for causal/surcausal. Third, we'll show it for policy stubs for pseudocausal/acausal, and finally we'll use that to show it for all partial policies for pseudocausal/acausal.

First, the easy direction. $↑ (Θ^{s t}) (π_{p a}) \supseteq ¯ ¯¯¯¯¯¯ ¯ c . h (⋃_{π \geq π_{p a}} p r_{*}^{π, π_{p a}} (↑ (Θ^{s t}) (π)))$

If we pick an arbitrary $M \in↑ (Θ^{s t}) (π)$ , it projects down to $Θ^{s t} (π_{s t})$ for all stubs $π_{s t}$ below $π$ . Since $π_{p a} \leq π$ , it projects down to all stubs beneath $π_{p a}$ . Since projections commute, $M$ projected down into $M^{a} (F (π_{p a}))$ makes a point that lies in the preimage of all the $Θ^{s t} (π_{s t})$ where $π_{s t} \leq π_{p a}$ , so it projects down into $↑ (Θ^{s t}) (π_{p a})$ .

This holds for all points in $↑ (Θ^{s t}) (π)$ , so $p r_{*}^{π, π_{p a}} (↑ (Θ^{s t}) (π)) \subseteq↑ (Θ^{s t}) (π_{p a})$ . This works for all $π \geq π_{p a}$ , so it holds for the union, and then due to closure and convexity which we've already shown, we get that the closed convex hull of the projections lies in $↑ (Θ^{s t}) (π_{p a})$ too, establishing one subset direction in full generality.

Now, for phase 2, deriving $↑ (Θ^{s t}) (π_{p a}) \subseteq ¯ ¯¯¯¯¯¯ ¯ c . h (⋃_{π \geq π_{p a}} p r_{*}^{π, π_{p a}} (↑ (Θ^{s t}) (π)))$ in the causal/surcausal case.

First, observe that if $π \geq π_{p a}$ , then $π^{n} \geq π_{p a}^{n}$ . Fix some $M \in↑ (Θ^{s t}) (π_{p a})$ and arbitrary $π \geq π_{p a}$ . We'll establish the existence of a $M^{'} \in↑ (Θ^{s t}) (π)$ that projects down to $M$ .

To begin with, $M$ projects down to $M_{n}$ in $Θ^{s t} (π_{p a}^{n})$ . Lock in a value for n, and consider the sequence that starts off $M_{0}, M_{1} . . . M_{n}$ , and then, by causality for stubs and $π^{n} \geq π_{p a}^{n}$ , you can find something in $Θ^{s t} (π^{n})$ that projects down onto $M_{n}$ , and something in $Θ^{s t} (π^{n + 1})$ that projects down onto that, and complete your sequence that way, making a sequence of points that all project down onto each other that climb up to $π$ . By Lemma 10, we get a $M_{n}^{'} \in↑ (Θ^{s t}) (π)$ . You can unlock n now. All these $M_{n}^{'}$ have the same $λ$ and $b$ value because projection preserves them, so we can isolate a convergent subsequence converging to some $M^{'} \in↑ (Θ^{s t}) (π)$ .

Assume $p r_{*}^{π, π_{p a}} (M^{'}) \neq M$ . Then we've got two different points. They differ at some finite stage, so there's some n where can project down onto $M^{a} (F (π_{p a}^{n}))$ to witness the difference, but from our construction process for $M^{'}$ , both $M^{'}$ and $M$ project down to $M_{n}$ , and we get a contradiction.

So, since $p r_{*}^{π, π_{p a}} (↑ (Θ^{s t}) (π)) =↑ (Θ^{s t} (π_{p a}))$ , this establishes the other direction, showing equality, and thus consistency, for causal/surcausal hypotheses.

For part 3, we'll solve the reverse direction for pseudocausal/acausal hypotheses in the case of stubs, getting $↑ (Θ^{s t}) (π_{s t}) \subseteq ¯ ¯¯¯¯¯¯ ¯ c . h (⋃_{π \geq π_{p a}} p r_{*}^{π, π_{s t}} (↑ (Θ^{s t}) (π)))$

Since we're working in the Nirvana-free case and are working with stubs, we can wield Lemma 3

$c . h ((Θ^{s t} (π_{s t}))^{min}) = c . h ((Θ^{s t} (π_{s t}))^{xmin})$

So, if we could just show that the union of the projections includes all the extreme minimal points, then when we take convex hull, we'd get the convex hull of the extreme minimal points, which by Lemma 3, would also nab all the minimal points as well. By Lemmas 11 and 12, our resulting convex hull of a union of projections from above would be upper-complete. It would also get all the minimal points, so it'd nabs the entire $Θ^{s t} (π_{s t})$ within it and this would show the other set inclusion direction for pseudocausal/acausal stubs. Also, we've shown enough to invoke Lemma 20 to conclude that said convex hull is closed. Having fleshed out that argument, all we need that all extreme minimal points are captured by the union of the projections.

By our previously proved extreme minimal point condition, for every extreme minimal point $M$ in $Θ^{s t} (π_{s t})$ , there's some $π > π_{s t}$ and $M^{'}$ in $↑ (Θ^{s t}) (π)$ that projects down to $M$ , which shows that all extreme points are included, and we're good.

For part 4, we'll show that in the nirvana-free pseudocausal/acausal setting, we have

$↑ (Θ^{s t}) (π_{p a}) \subseteq ¯ ¯¯¯¯¯¯ ¯ c . h (⋃_{π \geq π_{p a}} p r_{*}^{π, π_{p a}} (↑ (Θ^{s t}) (π)))$

Fix some arbitrary $M \in↑ (Θ^{s t}) (π_{p a})$ . Our task is to express it as a limit of some sequence of points that are mixtures of stuff projected from above.

For this, we can run through the same exact proof path that was used in the part of the Lemma 21 proof about how the nirvana-free part of $Θ (π_{p a})$ is a subset of closed convex hull of the projections of the nirvana-free parts of $Θ (π)$ , $π \geq π_{p a}$ . Check back to it. Since we're working in the Nirvana-free case, we can apply it very straightforwardly. The stuff used in that proof path is the ability to project down and land in $↑ (Θ^{s t}) (π_{p a}^{n})$ (we have that by how we defined $↑$ ), Hausdorff-continuity (which we have), and stubs being the convex hull, not the closed convex hull, of projections of stuff above them (which we mentioned recently in part 3 of our consistency proof).

Thus, consistency is shown.

Condition 6: Normalization.

Ok, now that we have all this, we can tackle our one remaining condition, normalization. Then move on to the two optional conditions, causality and pseudocausality.

A key thing to remember is, in this setting, when you're doing $E_{H} (f)$ , it's actually $E_{H \cap N F} (f)$ , because if Murphy picked a thing with nirvana in it, you'd get infinite value, which is not ok, so Murphy always picks a nirvana-free point.

Let's show the first part, that ${inf}_{π} E_{↑ (Θ^{s t}) (π)} (0) = 0$ . This unpacks as: ${inf}_{π} {min}_{(λ μ, b) \in↑ (Θ^{s t}) (π) \cap N F} b = 0$ and we have that ${inf}_{π_{s t}} {min}_{(λ μ, b) \in Θ^{s t} (π_{s t}) \cap N F} b = 0$

Projections preserves $b$ values, so we can take some nirvana-free point with a $b$ value of nearly 0, and project it down to $Θ^{s t} (π_{\emptyset})$ (belief function of the empty policy-stub) where there's no nirvana possible because no events happen.

So, we've got $b$ values of nearly zero in there. Do we have a point with a $b$ value of exactly zero? Yes. it's closed, and has bounded minimals, so we can go "all positive functionals are minimized by a minimal point", to get a point with a $b$ value of exactly zero. Then, we can invoke Lemma 21 (we showed consistency, extreme points, and all else required to invoke it) to decompose our point into the projections of nirvana-free stuff from above, all of which must have a $b$ value of 0. So, there's a nirvana-free point in some policy with 0 $b$ value.

Now for the other direction. Let's show that ${sup}_{π} E_{↑ (Θ^{s t}) (π)} (1) = 1$ .

This unpacks as: ${sup}_{π} {min}_{(λ μ, b) \in↑ (Θ^{s t}) (π) \cap N F} (λ + b) = 1$

We'll show this by disproving that the sup is <1, and disproving that the sup is >1.

First, assume that, regardless of $π$ , ${min}_{(λ μ, b) \in↑ (Θ^{s t}) (π) \cap N F} (λ + b) \leq 1 - ϵ$ Then, regardless of $π_{s t}$ we can pick some totally arbitrary $π > π_{s t}$ , and there's a nirvana-free point with a $λ + b$ value of $1 - ϵ$ or less. By consistency, we can project it down into $Θ (π_{s t})$ , to get a nirvana-free point with a $λ + b$ value of $1 - ϵ$ or less. Thus, regardless of the stub we pick, there's nirvana-free points where Murphy can force a value of $1 - ϵ$ or less, which contradicts ${sup}_{π_{s t}} {min}_{(λ μ, b) \in Θ^{s t} (π_{s t}) \cap N F} (λ + b) = 1$

What if it's above 1? Assume there's some $π$ where ${min}_{(λ μ, b) \in↑ (Θ^{s t}) (π) \cap N F} (λ + b) \geq 1 + ϵ$

From uniform-continuity-Hausdorff, pick some $δ$ to get a $\frac{ϵ}{2}$ Hausdorff-distance or lower (for stuff obeying the $λ^{⊙} + b^{⊙}$ bound, which all minimal points of $↑ (Θ^{s t}) (π) \cap N F$ do for all $π$ ). This $δ$ specifies some extremely large n, consider $π^{n}$ . Now, consider the set of every policy $π^{'}$ above $π^{n}$ . All of these are $δ$ or less away from $π$ . Also, remember that the particular sort of preimage-to-infinity that we used for Hausdorff-continuity slices away all the nirvana.

So, Murphy, acting on $π$ , can only force a value of $1 + ϵ$ or higher. Now, there can be no nirvana-free point in $↑ (Θ^{s t}) (π^{'})$ with $λ + b < 1 + \frac{ϵ}{2}$ . The reason for this is that, since $π^{'}$ is $δ$ or less away from $π$ , there's a nirvana-free point in $↑ (Θ^{s t}) (π)$ that's $\frac{ϵ}{2}$ away, and thus has $λ + b < 1 + ϵ$ , which is impossible.

Ok, so all the nirvana-free points in $↑ (Θ^{s t}) (π^{'})$ where $π^{'} > π_{s t}^{n}$ have $λ + b \geq 1 + \frac{ϵ}{2}$ .

Now, since we have Lemma 21, we can go "hm, $Θ^{s t} (π^{n})$ equals the convex hull of the projections of $↑ (Θ^{s t}) (π^{'}) \cap N F$ . Thus, any minimal point with $λ + b \leq 1$ is a finite mix of nirvana-free stuff from above, one of which must have $λ + b \leq 1$ . But we get a contradiction with the fact that there's no nirvana-free point from $π^{'}$ above $π^{n}$ with a $λ + b$ value that low, they're all $\geq 1 + \frac{ϵ}{2}$ "

So, since we've disproved both cases, ${sup}_{π} E_{↑ (Θ^{s t}) (π)} (1) = 1$ . And we're done with normalization! On to causality and pseudocausality.

Condition C: Causality.

An "outcome function" $o f$ for $↑ (Θ^{s t}) (π_{p a})$ is a function that maps a $π_{p a}$ to a point in $↑ (Θ^{s t}) (π_{p a})$ , s.t. for all $π_{p a}^{h i}, π_{p a}^{l o} : p r_{*}^{π_{p a}^{h i}, π_{p a}^{l o}} (o f (π_{p a}^{h i})) = o f (π_{p a}^{l o})$ .

Causality is, if you have a $M \in↑ (Θ^{s t}) (π_{p a})$ , you can always find an outcome function $o f$ where $o f (π_{p a}) = M$ . Sadly, all we have is causality over stubs. We'll be using the usual identification between $↑ (Θ^{s t}) (π_{s t})$ and $Θ^{s t} (π_{s t})$ .

Anyways, fix a $π_{p a}$ and a point $M$ in $↑ (Θ^{s t}) (π_{p a})$ . Project $M$ down to get a sequence $M_{n} \in Θ^{s t} (π_{p a}^{n})$ . By causality for stubs, we can find an $o f_{n}$ where, for all $π_{s t}$ , $o f_{n} (π_{p a}^{n}) = M_{n}$ . Observe that there are countably many stubs, and no matter the n, all the $λ$ and $b$ values are the same because projection preserves those. We can view $o f_{n}$ as a sequence in

$\prod_{π_{s t}} (Θ^{s t} (π_{s t}) \cap {(λ^{'} μ, b^{'}) | λ^{'} = λ \land b^{'} = b})$

By stub closure, and a $λ$ and $b$ bound, this is a product of compact sets, and thus compact by Tychonoff (no axiom of choice needed, its just a countable product of compact metric spaces) so we can get a limiting $o f^{s t}$ (because it's only defined over stubs).

An outcome function for stubs fixes an outcome function for all partial policies, by

$o f (π_{p a}) = ⋂_{n} (p r_{*}^{π_{p a}, π_{p a}^{n}})^{- 1} (o f^{s t} (π_{p a}^{n}))$

We've got several things to show now. We need to show that $o f^{s t}$ is an outcome function, that $o f$ is well-defined, that $o f (π_{p a}) = M,$ and that it's actually an outcome function.

For showing that $o f^{s t}$ is an outcome function, observe that projection is continuous, and, letting n index our convergent subsequence of interest, regardless of stub $π_{s t}$ , ${lim}_{n \to \infty} o f_{n} (π_{s t}) = o f^{s t} (π_{s t})$ . With this,

$p r_{*}^{π_{s t}^{h i}, π_{s t}^{l o}} (o f^{s t} (π_{s t}^{h i})) = p r_{*}^{π_{s t}^{h i}, π_{s t}^{l o}} ({lim}_{n \to \infty} o f_{n} (π_{s t}^{h i})) = {lim}_{n \to \infty} p r_{*}^{π_{s t}^{h i}, π_{s t}^{l o}} (o f_{n} (π_{s t}^{h i}))$

$= {lim}_{n \to \infty} o f_{n} (π_{s t}^{l o}) = o f^{s t} (π_{s t}^{l o})$

Now, let's show that $o f$ is well-defined. Since $o f^{s t}$ is an outcome function, all the points project down onto each other, so we can invoke Lemma 9 to show that the preimage is nonempty. If the preimage had multiple points, we could project down to some finite stage to observe their difference, but nope, they always project to the same point. So it does pick out a single well-defined point, and it lies in $↑ (Θ^{s t}) (π_{p a}$ ) by being a subset of the defining sequence of intersection of preimages.

Does $o f (π_{p a}) = M$ ? Well, $M$ projected down to all the $M_{n}$ . If $n \geq m$ , then $o f_{n} (π_{p a}^{m}) = p r_{*}^{π_{p a}^{n}, π_{p a}^{m}} (o f_{n} (π_{p a}^{n})) = p r_{*}^{π_{p a}^{n}, π_{p a}^{m}} (M_{n}) = M_{m}$ So, the limit specification $o f^{s t}$ has $o f^{s t} (π_{p a}^{n}) = M_{n}$ for all n. The only thing that projects down to make all the $M_{n}$ is $M$ itself, so $o f (π_{p a}) = M$ .

Last thing to check: Is $o f$ an outcome function over partial policies? Well, if $π_{p a}^{h i} \geq π_{p a}^{l o}$ , then for all n, $π_{p a}^{h i, n} \geq π_{p a}^{l o, n}$ . Assume $p r_{*}^{π_{p a}^{h i}, π_{p a}^{l o}} (o f (π_{p a}^{h i})) \neq o f (π_{p a}^{l o})$ . Then, in that case, we can project down to some $π_{p a}^{l o, n}$ and they'll still be unequal. However, since projections commute, it doesn't matter whether you project down to $π_{p a}^{l o}$ and then to $π_{p a}^{l o, n}$ , or whether you project down to $π_{p a}^{h i, n}$ (making $o f^{s t} (π_{p a}^{h i, n})$ ), and then project down to $π_{p a}^{l o, n}$ (making $o f^{s t} (π_{p a}^{l o, n})$ ). Wait, hang on, this is the exact point that $o f (π_{p a}^{l o})$ projects down to, contradiction. Therefore it's an outcome function.

And we're done, we took an arbitrary $π_{p a}$ and $M \in↑ (Θ^{s t}) (π_{p a})$ , and got an outcome function $o f$ with $o f (π_{p a}) = M$ , showing causality.

Condition P: Pseudocausality: If $(m, b) \in↑ (Θ^{s t}) (π_{p a})$ , and $m$ 's support is on $M^{a} (F^{N F} (π_{p a}^{'}))$ , then $(m, b) \in↑ (Θ^{s t}) (π_{p a}^{'})$ .

But all we have is, if $(m, b) \in Θ^{s t} (π_{s t})$ and $m$ 's support is on $M^{a} (F (π_{s t}^{'}))$ , then $(m, b) \in Θ^{s t} (π_{s t}^{'})$ .

There's a subtlety here. Our exact formulation of pseudocausality we want is the condition $supp (m) \subseteq F (π_{p a}^{'})$ , so if the measure is 0, then support is the empty set, which is trivially a subset of everything, then pseudocausality transfers it to all partial policies.

Ok, so let's assume that $M \in↑ (Θ^{s t}) (π_{p a})$ , and the measure part $m$ has its support being a subset of $M^{a} (F^{N F} (π_{p a}^{'}))$ but yet is not in $↑ (Θ^{s t}) (π_{p a}^{'})$ . Then, since this is an intersection of preimages from below, there should be some finite level $π_{p a}^{^{'} n}$ that you can project $M$ down to (it's present in $M^{a} (F^{N F} (π_{p a}^{'}))$ , just maybe not in $↑ (Θ^{s t}) (π_{p a}^{'})$ ) where the projection of $M$ (call it $M_{n}^{'}$ ) lies outside $Θ^{s t} (π_{p a}^{^{'} n})$ (lying outside the intersection of preimages)

This is basically "take $m$ , chop it off at height n". However, since $M \in↑ (Θ^{s t}) (π_{p a})$ , you can project it down to $Θ^{s t} (π_{p a}^{n})$ . Which does the exact same thing of chopping $m$ off at height n, getting you $M_{n}^{'}$ exactly. We can invoke stub-pseudocausality (because with full measure, the history will land in $F (π_{p a}^{'})$ , then with full measure, the truncated history will land in $F (π_{p a}^{^{'} n})$ as the latter is prefixes of the former, or maybe the full measure is 0 in which case pseudocausality transfer still works) to conclude that $M^{'}$ actually lies inside $Θ^{s t} (π_{p a}^{^{'} n})$ , getting a contradiction. This establishes pseudocausality in full generality.

Ok, so we have one direction. $↑ (Θ^{s t})$ is a hypothesis, if $Θ^{s t}$ fulfills analogues of the hypothesis conditions for the finitary stub case. Our proof of everything doesn't distinguish between causal and surcausal, and the arguments work for all types of hypotheses, whether causal, surcausal, pseudocausal, or acausal. Ok, we're 1/4 of the way through. Now we do the same thing, but for building everything from infinitary hypotheses.

PART 2: Infinitary hypotheses. We now consider $↓ (Θ^{ω})$ , defined as $↓ (Θ^{ω}) (π_{p a}) = ¯ ¯¯¯¯¯¯ ¯ c . h (⋃_{π \geq π_{p a}} p r_{*}^{π, π_{p a}} (Θ^{ω} (π))$

We'll show that with 8+2 defining conditions, the 9+2 defining conditions for a hypothesis hold for $↓ (Θ^{ω})$ . The the 8+2 conditions for a $Θ^{ω}$ are:

1: Infinitary Nirvana-Free Nonemptiness: $\forall π : Θ^{ω} (π) \cap N F \neq \emptyset$

2: Infinitary Closure: $\forall π : Θ^{ω} (π) = ¯ ¯¯¯¯¯¯¯¯¯¯¯¯ ¯ Θ^{ω} (π)$

3: Infinitary Convexity. $\forall π : Θ^{ω} (π) = c . h (Θ^{ω} (π))$

4: Infinitary Nirvana-free Upper-Completeness

$\forall π : Θ^{ω} (π) \cap N F = ((Θ^{ω} (π) \cap N F) + M^{s a} (F^{N F} (π))) \cap M^{a} (F (π))$

5: Infinitary Bounded Minimals: $\exists λ^{⊙}, b^{⊙} \forall π : (λ μ, b) \in {(Θ^{ω} (π))}^{min} \to λ + b \leq λ^{⊙} + b^{⊙}$

6: Normalization: ${inf}_{π} E_{Θ^{s t} (π)} (0) = 0$ and ${sup}_{π} E_{Θ^{s t} (π)} (1) = 1$

7: Nirvana-free consistency.

$\forall π_{s t} : c . h (⋃_{π > π_{s t}} p r_{*}^{π, π_{s t}} (Θ^{ω} (π) \cap N F)) = ¯ ¯¯¯¯¯¯ ¯ c . h (⋃_{π > π_{s t}} p r_{*}^{π, π_{s t}} (Θ^{ω} (π))) \cap N F$

8: Infinitary Uniform Hausdorff Continuity:

The function $π \mapsto Θ^{ω} (π) \cap N F \cap {\leq ⊙}$ is uniformly continuous.

C: Infinitary Causality: Regardless of $π$ and $M \in Θ^{ω} (π)$ , there's an outcome function $o f$ over full policies s.t. $o f (π) = M$ , and for all $π^{'}$ and $π^{''}$ , $p r_{*}^{π^{'}, inf (π^{'}, π^{''})} (o f (π^{'})) = p r_{*}^{π^{''}, inf (π^{'}, π^{''})} (o f (π^{''}))$

P: Infinitary Pseudocausality: $\forall π, π^{'} : (M \in Θ^{ω} (π), supp (M) \subseteq F^{N F} (π^{'}) \to π_{s t} \in Θ^{ω} (π^{'})$

Let's begin showing the conditions.

Condition 1: Nirvana-free Nonemptiness.

This one is trivial. Pick some $π \geq π_{p a}$ . There's a nirvana-free point. Project it down. You get a nirvana-free point and you're done.

Conditions 2 and 3, Closure and convexity. We explicitly took the closed convex hull when defining everything, these are tautological.

Condition 4: Nirvana-free upper completion.

For the pseudo/acausal case, it's doable by Lemmas 10, 11, and 12. The projection of an upper-complete set (by infinitary nirvana-free upper-completion) is upper-complete, so the union of projections is upper-complete, and then the convex hull is upper-complete, and then the closure is upper-complete and we're done.

We'll have to loop back to the causal case of Nirvana-free Upper Completion later, because we need Lemma 21 to make it go through and that requires consistency and the extreme point condition to make it work.

Condition 5: Bounded Minimals.

We can break down into three phases. First is showing that all points in the projection set have something under them that respects the $λ^{⊙} + b^{⊙}$ bound. Second is showing that all points in the convex hull of the union of projection sets have something under them that respects the $λ^{⊙} + b^{⊙}$ bound. Third is showing that all points in the closure have something under them that respects the usual bound. The reason we have to phrase it this way is that we don't necessarily know that our sets of interest are closed until the end, so we can't find a minimal point, just a bounded one that is lower, but that suffices to show that a "minimal point" that violates the restricted minimal condition isn't actually minimal.

For part 1, let $M \in p r_{*}^{π, π_{p a}} (Θ^{ω} (π))$ . Then, $M$ is the projection of some point $M^{'} \in Θ^{ω} (π)$ . By infinitary bounded-minimals, we can find a minimal point $M^{min} \in Θ^{ω} (π)$ below $M^{'}$ that obeys the $λ^{⊙} + b^{⊙}$ bound, so $M^{min} + M^{*} = M^{'}$ . Projecting down is linear, so we get $p r_{*}^{π, π_{p a}} (M^{min}) + p r_{*}^{π, π_{p a}} (M^{*}) = M$ , and $p r_{*}^{π, π_{p a}} (M^{min})$ is below $M$ and fulfills the $λ^{⊙} + b^{⊙}$ bound.

For part 2, let $M \in c . h (⋃_{π \geq π_{p a}} p r_{*}^{π, π_{p a}} (Θ^{ω} (π)))$ We can rewrite $M$ as $E_{ζ} M_{i}$ , and then, by part 1, decompose the $M_{i}$ into $M_{i}^{l o}$ (not actually minimal, just a point obeying the $λ^{⊙} + b^{⊙}$ bound) and $M_{i}^{*}$ . Then we can decompose $M$ further into $E_{ζ} M_{i}^{l o} + E_{ζ} M_{i}^{*}$ . The former is an a-measure (mix of a-measures) and obeys the $λ^{⊙} + b^{⊙}$ bound since all its components do, and it's in the relevant convex hull, witnessing that $M$ has a point below it in the convex hull that obeys the bounds.

For part 3, let $M$ be in the closure of the convex hull. There's some sequence $M_{n}$ in the convex hull that limits to $M$ . Below each $M_{n}$ we can find a $M_{n}^{l o}$ (again, not actually minimal) that obeys the $λ^{⊙} + b^{⊙}$ bound. Invoke Lemma 16 to get a point below $M$ that respects the bounds, and we're done.

Condition 6: Normalization.

We literally have the exact phrasing of normalization we need already, this is a tautology.

Condition 7: Consistency.

Ok, one direction is trivial because $↓ (Θ^{ω}) (π) = Θ^{ω} (π)$ , so we can just use the definition of $↓$ . $↓ (Θ^{ω}) (π_{p a}) = ¯ ¯¯¯¯¯¯ ¯ c . h (⋃_{π \geq π_{p a}} p r_{*}^{π, π_{p a}} (↓ (Θ^{ω}) (π)))$

The other direction, that everything equals the intersection of preimages of stuff below it, is trickier. One subset direction isn't too bad, the one that

$↓ (Θ^{ω}) (π_{p a}) \subseteq ⋂_{π_{s t} \leq π_{p a}} (p r_{*}^{π_{p a}, π_{s t}})^{- 1} (↓ (Θ^{ω}) (π_{s t}))$

If we take a $M \in↓ (Θ^{ω}) (π_{p a})$ that wasn't added in the final closure step, it's expressible as $E_{ζ} M_{i}$ , and all the $M_{i}$ come from points $M_{i}^{\infty}$ in $Θ^{ω} (π_{i})$ where $π_{i} \geq π_{p a}$ . Projecting the $M_{i}^{\infty}$ down to a $π_{s t} \leq π_{p a}$ instead makes $M_{i}^{l o}$ , which mix together in the same way to make $M^{l o} \in↓ (Θ^{ω}) (π_{s t})$ . Because projections are linear and commute, $M_{l o}$ is the projection of $M$ . So, any point in $↓ (Θ^{ω}) (π_{p a})$ (without the closure step) projects down to lie in $↓ (Θ^{ω}) (π_{s t})$ for any $π_{s t} \leq π_{p a}$ .

Then, for the closure step, we just fix a sequence $M_{n}$ limiting to $M$ . The $M_{n}$ can project down to whichever $↓ (Θ^{ω}) (π_{s t})$ you wish, and by continuity of projection, the $M$ comes along for the ride as a limit point. However, $↓ (Θ^{ω}) (π_{s t})$ is closed, so $M$ projects down to land in that set as well. Bam, any old $M \in↓ (Θ^{ω}) (π_{p a})$ projects down to land in any $↓ (Θ^{ω}) (π_{s t})$ set you wish with $π_{s t} \leq π_{p a}$ , certifying that $↓ (Θ^{ω}) (π_{p a})$ lies in the intersection of preimages of stubs below.

Now, we just have to establish $↓ (Θ^{ω}) (π_{p a}) \supseteq ⋂_{π_{s t} \leq π_{p a}} (p r_{*}^{π_{p a}, π_{s t}})^{- 1} (↓ (Θ^{ω}) (π_{s t}))$ which splits into two cases. The causal/surcausal case, and the pseudocausal/acausal case where you don't have to worry about nirvana.

For the nirvana-free case... We can use the same proof strategy as the last part of Lemma 21, where we were showing the result for partial policies. It may be a bit nonobvious why it works. We do need to swap things around a bit, and will mention important changes without fleshing out the fiddly details, which are already given in the last part of Lemma 21.

Start with a $M$ in the intersection of preimages of stubs below. To show it's in $↓ (Θ^{ω}) (π_{p a})$ , we need a sequence limiting to it, where each member of the sequence is a mix of finitely many points projected down from policies above $π_{p a}$ . The end part of Lemma 21 gives how to construct such a sequence. The fact that we're working in a nirvana-free setting means you can ignore all fiddly details about points being nirvana-free and preimages of only the nirvana-free parts, because everything fulfills that. The key steps in that proof path are:

1: being able to project down $M$ to make a sequence $M_{n} \in↓ (Θ^{ω}) (π_{p a}^{n})$ . We trivially have this by $M$ being defined as "in the intersection of preimages of stubs below it".

2: Having uniform Hausdorff-continuity for the policies. This is our condition 8 we're assuming, so we're good there.

3: The ability to shatter our $M_{n}$ into finitely many $M_{i, n}$ which are the projections of various $M_{i, n}^{\infty}$ points from above. This is the key difference. The proof of Lemma 21 had to set up that fact beforehand. However, in our case, we have the Nirvana-free consistency condition, which says

$\forall π_{s t} : c . h (⋃_{π > π_{s t}} p r_{*}^{π, π_{s t}} (Θ^{ω} (π) \cap N F)) = ¯ ¯¯¯¯¯¯ ¯ c . h (⋃_{π > π_{s t}} p r_{*}^{π, π_{s t}} (Θ^{ω} (π))) \cap N F$

But, since we're working in the nirvana-free setting, this turns into:

$\forall π_{s t} : c . h (⋃_{π > π_{s t}} p r_{*}^{π, π_{s t}} (Θ^{ω} (π))) = ¯ ¯¯¯¯¯¯ ¯ c . h (⋃_{π > π_{s t}} p r_{*}^{π, π_{s t}} (Θ^{ω} (π)))$

And that right-hand term... is just the definition of $↓ (Θ^{ω}) (π_{s t})$ ! So, swapping that out, and specializing our arbitrary stub to $π_{p a}^{n}$ , we have:

$c . h (⋃_{π > π_{p a}^{n}} p r_{*}^{π, π_{p a}^{n}} (Θ^{ω} (π))) =↓ (Θ^{ω}) (π_{p a}^{n})$

So, since our $M_{n}$ lie in $↓ (Θ^{ω}) (π_{p a}^{n})$ , they can be written as a finite mix of nirvana-free things from above projected down, and the Lemma 21 argument goes through.

Now, for the nirvana cases where we can assume infinitary causality. We'll do this by showing a little sublemma, that, if $π \geq π_{p a}$ , then $p r_{*}^{π, π_{p a}} (Θ^{ω} (π)) =↓ (Θ^{ω}) (π_{p a})$

First, we'll show that if $π, π^{'} \geq π_{p a}$ , then $p r_{*}^{π, π_{p a}} (Θ^{ω} (π)) = p r_{*}^{π^{'}, π_{p a}} (Θ^{ω} (π^{'}))$

Fix an arbitrary $M$ in the projection of $Θ^{ω} (π)$ , we can get a preimage point $M^{h i} \in Θ^{ω} (π)$ . Then, by infinitary causality, we can make a point $M^{^{'} h i} \in Θ^{ω} (π^{'})$ that projects down to $M$ . Just make an outcome function $o f$ where $o f (π) = M^{h i}$ , feed in $π^{'}$ , that gets you your point $M^{^{'} h i}$ , the two agree when you project them down to $inf (π, π^{'})$ , and $π_{p a}$ is further down than that and projections commute so they both hit the same point $M$ if you project down. Flipping $π$ and $π^{'}$ shows our equality.

Alright, so now $↓ (Θ^{ω}) (π_{p a})$ can be written as $¯ ¯¯¯¯¯¯ ¯ c . h (p r_{*}^{π, π_{p a}} (Θ^{ω} (π)))$ where $π$ is arbitrary above $π_{p a}$ .

Projection is linear, so the projection of a convex set is convex. To get the closure points, just take a sequence $M_{n}$ in the projection limiting to some $M$ . Take preimage points $M_{n}^{\infty} \in Θ^{ω} (π)$ . There's a bound on the $λ$ and $b$ values of this sequence because projections preserve $λ$ and $b$ and our sequence $M_{n}$ converges, so we can apply the Compactness Lemma and get a convergent subsequence limiting to a point $M^{\infty}$ , which must be in $Θ^{ω} (π)$ because closure. Projection is continuous, so $M^{\infty}$ projects down to $M$ . And we have $p r_{*}^{π, π_{p a}} Θ^{ω} (π) =↓ (Θ^{ω}) (π_{p a})$ proved! Wow, that was a sublemma of case 2 of part 3 of the proof of condition 7 in part 2 of the proof of the Isomorphism theorem, we're really in the weeds at this point.

Moving on, how can we use this to show $↓ (Θ^{ω}) (π_{p a}) \supseteq ⋂_{π_{s t} \leq π_{p a}} (p r_{*}^{π_{p a}, π_{s t}})^{- 1} (↓ (Θ^{ω}) (π_{s t}))$ for the causal case, which is the last bit we need to show consistency?

Well, fix a $M$ in the intersection of preimages, and an arbitrary $π \geq π_{p a}$ . $M$ projects down to make some $M_{n} \in↓ (Θ^{ω}) (π_{p a}^{n})$ . Since $π \geq π_{p a} \geq π_{p a}^{n}$ , and we have our sublemma, there's a point $M_{n}^{\infty}$ in $Θ^{ω} (π)$ that projects down to some $M_{n}^{'} \in↓ (Θ^{ω}) (π_{p a})$ , and further down to $M_{n}$ .

This sequence $M_{n}^{\infty}$ all has the same $λ$ and $b$ value since projection preserves those, so by the Compactness Lemma and closure, there's a convergent subsequence and limit point $M^{\infty} \in Θ^{ω} (π)$ . Does $M^{\infty}$ project down onto $M$ ? (witnessing that $M \in↓ (Θ^{ω}) (π_{p a}))$ ?

Well, let's say it didn't and projecting down gets you a distinct point. Then there's some n where projecting down further to $π_{p a}^{n}$ would keep the points distinct, since they have to differ at some finite time. But... after time n, our sequence $M_{n}^{\infty}$ is roaming around entirely in the preimage of $M_{n}$ , so the limit point is in there too, and it projects down to $M_{n}$ and we have a contradiction. Therefore, $M^{\infty} \in Θ^{ω} (π)$ projects down onto $M$ , witnessing that $M \in↓ (Θ^{ω}) (π_{p a})$ , and $M$ was arbitrary in the intersection of preimages.

So, we have $↓ (Θ^{ω}) (π_{p a}) \supseteq ⋂_{π_{s t} \leq π_{p a}} (p r_{*}^{π_{p a}, π_{s t}})^{- 1} (↓ (Θ^{ω}) (π_{s t}))$ for the nirvana-containing causal/surcausal case, which is the last piece we needed to show consistency.

Condition 8: Extreme point condition.

The thing we want is that an extreme nirvana-free minimal point $M^{e x}$ in $↓ (B F^{ω}) (π_{s t}) \cap N F$ is the projection of a nirvana-free point from a policy above it. By the nirvana-free consistency property, $M^{e x}$ lies in $c . h (⋃_{π \geq π_{s t}} p r_{*}^{π, π_{s t}} (Θ^{ω} (π) \cap N F))$

$M^{e x}$ is extreme, so it lies in $⋃_{π \geq π_{s t}} p r_{*}^{π, π_{s t}} (Θ^{ω} (π) \cap N F)$

So, there's some point in some $Θ^{ω} (π) \cap N F$ that projects down to $M^{e x}$ and we're done.

Condition 9: Hausdorff Continuity:

Ok, this one is going to be complicated. We'll work with the Lemma 15 version of Hausdorff-continuity, where a $δ$ difference between two policies means that if you start off in one preimage, you've gotta travel $ϵ (1 + λ)$ distance or less to get to the preimage associated with the other policy, and vice-versa.

We split into two parts. Part 1 is showing that if $π_{p a} \geq π_{s t}$ , and the distance between $π_{p a}$ and $π_{s t}$ is $δ$ or less, the distance between their respective preimages is low. Part 2 is showing that if $π_{p a}$ and $π_{p a}^{'}$ are $δ$ apart, then we can exploit part 1 to get that the distance between the preimages is low, and will be pretty easy after we get part 1.

Our Hausdorff-continuity condition links $δ$ and $ϵ$ . So, when we fix an $ϵ$ and are like "how close do the policies have to be to guarantee the preimages are $ϵ$ apart", pick the $δ$ that gets you $\frac{ϵ}{3}$ distance w.r.t our original Hausdorff-continuity condition, and also have $δ < \frac{ϵ}{3}$ .

Our first part uses $M^{l o}$ and $M^{^{'} l o}$ for points in $↓ (Θ^{ω}) (π_{s t}) \cap N F$ , $M^{^{'} m i d}$ for a point in $↓ (Θ^{ω}) (π_{p a}) \cap N F$ , $M^{h i}$ and $M^{^{'} h i}$ for points in $M^{a} (\infty)$ (that are expressible as a finite mix of points from $Θ^{ω} (π)$ for varying $π$ ), and $M, M^{'}$ for two more general points in $M^{a} (\infty)$ .

So, one half of showing the two preimages are close to each other is trivial. Everything in $↓ (Θ^{ω}) (π_{p a}) \cap N F$ projects down into $↓ (Θ^{ω}) (π_{s t}) \cap N F$ by consistency, and projection preserving nirvana-freeness, so the preimage associated with $π_{s t}$ is a superset of the preimage associated with $π_{p a}$ , so there's distance 0 from a point in the $π_{p a}$ preimage to a point in the $π_{s t}$ preimage.

The other half is trickier. Pick an arbitrary point $M$ in $(p r_{*}^{\infty, π_{s t}})^{- 1} (↓ (Θ^{ω}) (π_{s t}) \cap N F)$ and $λ$ is the $λ$ value of this point. $M$ projects down to some point $M^{l o}$ in $↓ (Θ^{ω}) (π_{s t}) \cap N F$ . From nirvana-free consistency, $M \in c . h (⋃_{π \geq π_{s t}} p r_{*}^{π, π_{s t}} (Θ^{ω} (π) \cap N F))$ , $M^{l o}$ can then be produced by (keeping in mind that it doesn't matter whether we mix before or after projecting) finitely many points in varying $Θ^{ω} (π) \cap N F$ sets that are mixed to make a point $M^{h i}$ , and then projected down.

Important note: $M^{h i}$ is not necessarily equal to, or even close to, $M$ .

Because $π_{s t}$ is within $δ$ of $π_{p a}$ , every policy above $π_{s t}$ has a corresponding policy within $δ$ that lies above $π_{p a}$ . Thus, we can perturb the component points (indexed by i) that mix to make $M^{h i}$ by $\frac{ϵ}{3} (1 + λ_{i})$ (infinitary Hausdorff-uniform-continuity Lemma 15 variant, $δ$ was assumed to be small enough for that to be the case), and mix them, to get a $M^{^{'} h i}$ in $c . h (⋃_{π \geq π_{p a}} (Θ^{ω} (π) \cap N F))$

$M^{^{'} h i}$ projects down to $↓ (Θ^{ω}) (π_{p a}) \cap N F$ to make a $M^{^{'} m i d}$ , and further projects down to $↓ (Θ^{ω}) (π_{s t}) \cap N F$ to make a $M^{^{'} l o}$ . Because $M^{^{'} h i}$ is within $\frac{ϵ}{3} (1 + λ)$ of $M^{h i}$ , projecting down (and projecting being nonexpansive) means that $M^{^{'} l o}$ is within $\frac{ϵ}{3} (1 + λ)$ of $M^{l o}$ .

Now, we can take $M^{^{'} l o}$ and fill in all the missing measure data to get a $M^{'}$ that projects down onto $M^{^{'} m i d}$ (certifying that it's in the preimage of $↓ (Θ^{ω}) (π_{p a}) \cap N F)$ as follows. Our most important constraint is that, when extending $m^{^{'} l o}$ , it should perfectly mimic $m^{^{'} m i d}$ so it can project down onto it. Our second constraint is that, if $m^{l o}$ doesn't specify what comes after a finite history and it doesn't conflict with the first constraint, it should exactly mimic the conditional probabilities of $m$ . Also, our $δ$ fixes a first time n (ie, ${log}_{γ} (δ)$ ) at which $π_{p}$