# Formal Open Problem in Decision Theory

AI Alignment Forum

# Ω 12

(This post was originally published on March 31st 2017, and has been brought forwarded as part of the AI Alignment Forum launch sequence on fixed points.)

In this post, I present a new formal open problem. A positive answer would be valuable for decision theory research. A negative answer would be helpful, mostly for figuring out what is the closest we can get to a positive answer. I also give some motivation for the problem, and some partial progress.

Open Problem: Does there exist a topological space (in some convenient category of topological spaces) such that there exists a continuous surjection from to the space (of continuous functions from to )?

Motivation:

Topological Naturalized Agents: Consider an agent who makes some observations and then takes an action. For simplicity, we assume there are only two possible actions, and . We also assume that the agent can randomize, so we can think of this agent as outputting a real number in , representing its probability of taking action .

Thus, we can think of an agent as having a policy which is a function from the space of possible observations to . We will require that our agent behaves continuously as a function of its observations, so we will think of the space of all possible policies as the space of continuous functions from to , denoted .

We will let denote the space of all possible agents, and we will have a function which takes in an agent, and outputs that agent's policy.

Now, consider what happens when there are other agents in the environment. For simplicity, we will assume that our agent observes one other agent, and makes no other observations. Thus, we want to consider the case where , so .

We want to be continuous, because we want a small change in an agent to correspond to a small change in the agent's policy. This is particularly important since other agents will be implementing continuous functions on agents, and we would like any continuous function on policies to be able to be considered valid continuous function on agents.

We also want to be surjective. This means that our space of agents is sufficiently rich that for any possible continuous policy, there is an agent in our space that implements that policy.

In order to meet all these criteria simultaneously, we need a space of agents, and a continuous surjection .

Unifying Fixed Point Theorems: While we are primarily interested in the above motivation, there is another secondary motivation, which may be more compelling for those less interested in agent foundations.

There are (at least) two main clusters of fixed point theorems that have come up many times in decision theory, and mathematics in general.

First, there is the Lawvere cluster of theorems. This includes the Lawvere fixed point theorem, the diagonal lemma, and the existence of Quines and fixed point combinators. These are used to prove Gödel's incompleteness Theorem, Cantor's Theorem, Löb's Theorem, and achieve robust cooperation in the Prisoner's Dilemma in modal framework and bounded variants. All of these can be seen as corollaries of Lawvere's fixed point theorem, which states that in a cartesian closed category, if there is a point-surjective map , then every morphism has a fixed point.

Second, there is the Brouwer cluster of theorems. This includes Brouwer's fixed point theorem, The Kakutani fixed point theorem, Poincaré–Miranda, and the intermediate value theorem. These are used to prove the existence of Nash Equilibria, Logical Inductors, and Reflective Oracles.

If we had a topological space and a continuous surjection , this would allow us to prove the one-dimensional Brouwer fixed point theorem directly using the Lawvere fixed point theorem, and thus unify these two important clusters.

Thanks to Qiaochu Yuan for pointing out the connection to Lawvere's fixed point theorem (and actually asking this question three years ago).

Partial Progress:

Most Diagonalization Intuitions Do Not Apply: A common initial reaction to this question is to conjecture that such an does not exist, due to cardinality or diagonalization intuitions. However, note that all of the diagonalization theorems pass through (some modification of) the same lemma: Lawvere's fixed point theorem. However, this lemma does not apply here!

For example, in the category of sets, the reason that there is no surjection from any set to the power set, , is because if there were such a surjection, Lawvere's fixed point theorem would imply that every function from to itself has a fixed point (which is clearly not the case, since there is a function that swaps and ).

However, we already know by Brouwer's fixed point theorem that every continuous function from the interval to itself has a fixed point, so the standard diagonalization intuitions do not work here.

Impossible if You Replace with e.g. : This also provides a quick sanity check on attempts to construct an . Any construction that would not be meaningfully different if the interval is replaced with the circle is doomed from the start. This is because a continuous surjection would violate Lawvere's fixed point theorem, since there is a continuous map from to itself without fixed points.

Impossible if you Require a Homeomorphism: When I first asked this question I asked for a homeomorphism between and . Sam Eisenstat has given a very clever argument why this is impossible. You can read it here. In short, using a homeomorphism, you would be able to use Lawvere to construct a continuous map that send a function from to itself to a fixed point of that function. However, no such continuous map exists.

Notes:

If you prefer not to think about the topology of , you can instead find a space , and a continuous map , such that for every continuous function , there exists an , such that for all , .

Many of the details in the motivation could be different. I would like to see progress on similar questions. For example, you could add some computability condition to the space of functions. However, I am also very curious which way this specific question will go.

This post came out of many conversations, with many people, including: Sam, Qiaochu, Tsvi, Jessica, Patrick, Nate, Ryan, Marcello, Alex Mennen, Jack Gallagher, and James Cook.

This post was originally published on March 31st 2017, and has been brought forwarded as part of the AI Alignment Forum launch sequences.

Tomorrow's AIAF sequences post will be 'Iterated Amplification and Distillation' by Ajeya Cotra, in the sequence on iterated amplification.

New Comment
[-]RupertΩ260

I have just now submitted an attempted solution to this problem to "Geometry and Topology". I claim that the space you are looking for is ( being the least uncountable cardinal) with the generalised Cantor space topology", that is for each countable well-ordered bit-string you have a basic open set consisting of all bit-strings of length with as an initial fragment. Since this topological space has quite a large cardinality I'm somewhat unclear whether this is helpful for your proposed application and would need to think about it more. (Matthew Barnett just now directed me to this post of yours.) I sent you an early draft of my paper, which argues the point in detail, on FB Messenger, and can send the latest version to you if you wish.

[-]RupertΩ110

Let be with generalised Cantor space topology, and be with product topology, a closed disc in a finite-dimensional Euclidean space. Then there is a continuous surjection . I don't know how to show that there is a topological space with carrier set and a continuous surjection . Thanks to Alex Mennen for pointing out the problem.

[-]RupertΩ110

However, because topology on is finer than topology on here, this still shows how the proof of the Lawvere fixed point theorem can be applied here to give Brouwer fixed point theorem as corollary, which could conceivably be a publishable result (see what "Geometry and Topology" think about that), and this could still be sorta kinda maybe relevant to Scott's original motivation for looking at the problem (if you're okay with working with two different topologies on the space of agents, one finer than the other). But this is a very big space of agents you're talking about here.

Correction: need not only that topology on is finer than topology on , but also, given arbitrary open subset of , take pre-image under evaluation map in , projection onto first factor and then pre-image of that under the continuous surjection , it needs to be shown that this set is open in both topologies. I believe that this can indeed be done for an appropriate class of spaces for the pair of topologies in question.

[-]RupertΩ110

When I look at my post the LaTeX code isn't formatting properly; if anyone can let me know how to fix that.

I fixed it. In our editor, use cmd-4/ctrl-4 to do LaTex, not dollar signs. (The thing you did would work in the markdown editor – you can go into settings to change to that editor if you'd like.)

From discussions I had with Sam, Scott, and Jack:

To solve the problem, it would suffice to find a reflexive domain with a retract onto .

This is because if you have a reflexive domain , that is, an with a continuous surjective map , and is a retract of , then there's also a continuous surjective map .

Proof: If is a retract of then we have a retraction and a section with . Construct . To show that is a surjection consider an arbitrary . Thus, . Since is a surjection there must be some with . It follows that . Since was arbitrary, is also a surjection.

A small note: it's not hard to construct spaces that are a bit too big, or a bit too small (raising the possibility that a true lies between them).

For instance, if is the unit interval, then we can map onto the countable hypercube ( https://en.wikipedia.org/wiki/Space-filling_curve#The_Hahn.E2.80.93Mazurkiewicz_theorem ). Then if we pick an ordering of the dimensions of the hypercube and an ordering of , we can see any element of - hence any element of - as a function from to .

Let be the space of continuous functions . Then any element of defines a unique function (the converse is not true - most functions do not correspond to continuous functions ). Pulling back to via we define the set .

Thus maps surjectively onto . However, though maps into by restriction (any function from is a function from ), this map is not onto (for example, there are more continuous functions from than there are from , because of the potential discontinuity at ).

Now, there are elements of that map to functions in but not in . So there's a hope that there may exist an with , , and mapping onto . Basically, as `gets bigger', its image in grows, while itself shrinks, and hopefully they'll meet.

I think you made a mistake here

The Hahn–Mazurkiewicz theorem states that

A non-empty Hausdorff topological space is a continuous image of the unit interval if and only if it is a compact, connected, locally connected second-countable space.

I will agree that is connected and locally connected. I'm not sure if its second countable. It is not compact.

Just to be clear Now let . Clearly each is open. Let And . Now clearly this family covers all of . However, remove any from and is no longer covered. So is a family of open sets, which cover and don't have any finite subcover.

Hum, should be compact by Tychonoff's theorem (see also the Hilbert Cube, which is homeomorphic to ).

For your proof, I think that is not open in the product topology. The product topology is the coarsest topology where all the projection maps are continuous.

To make all the projection maps continuous we need all sets in to be open, where we define iff there exists an , such that is open in and .

Let be the set of finite intersection of these sets. For any , there exists a finite set such that if and for , then as well.

If we take to be the arbitrary union of , this condition will be preserved. Thus is not contained in the arbitrary unions and finite intersections of , so it seems it is not an open sent.

Also, is second-countable. From the wikipedia article on second-countable:

Any countable product of a second-countable space is second-countable

I've figured out the difference, I was using the box topology https://en.wikipedia.org/wiki/Box_topology , while you were using the https://en.wikipedia.org/wiki/Product_topology.

You are correct. I knew about finite topological products and made a natural generalization, but it turns out not to be the standard meaning of .

Thanks for introducing me to the box topology - seeing it defined so explicitly, and seeing what properties it fails, cleared up a few of my intuitions.

If I were studying this, I would be looking at domain theory, in which (among other things) there has been found a topological space homeomorphic with . The page I linked links to some notes at the bottom. (h/t Qiaochu for pointing out domain theory)

When thinking about agents, the first motivation might not quite work out. Small changes in observation might introduce discontinuous changes in policy - e.g. in the Matching Pennies game. Suppose there are agents (functions) in that output a fixed , no matter their input. If you can continuously vary by moving in , then Matching Pennies play will be discontinuous at . So right away you've committed to some unusual behavior for the agents in by asking for continuity - they can't play perfect Matching Pennies at the very least.

EDIT: This idea is wrong: https://www.lesswrong.com/posts/eqi83c2nNSX7TFSfW/no-surjection-onto-function-space-for-manifold-x

Ok, here's an idea for constructing such a map, with a few key details left unproven; let me know if people see any immediate flaws in the approach, before I spend time filling in the holes.

Let be a countable collection of open intervals (eg ), given the usual topology. Let be the closed unit interval, and the set of continuous functions from to . Give the compact-open topology.

By the properties of the compact-open topology, since is (Tychonoff), then so is . I'm hoping that the proof can be extended, at least in this case, to show that is (normal Haussdorff).

It seems clear that is second-countable: let consist of all functions that map into , where is the intersection of with a closed interval with rational endpoints, and is an open interval with rational endpoints. The set of all such is countable, and forms a subbasis of . A countable subbasis means a countable basis, as the set of finite subsets of countable set, is itself countable.

If is and second countable, then it is homeomorphic to a subset of the Hilbert Cube. To simplify notation, we will identify with its image in the Hilbert Cube.

Take the closure of within the Hilbert Cube. This closure is compact and second countable (since the Hilbert Cube itself is both). It seems clear that is connected and locally connected; connected will extend to the closure, we'll need to prove that locally connected does as well.

Then we can apply the Hahn-Mazurkiewicz theorem:

• A non-empty Hausdorff topological space is a continuous image of the unit interval if and only if it is a compact, connected, locally connected second-countable space.

So there is a continuous surjection . Pull back , defining . If is open in (with the subspace topology), then is open in . Even if is not open, we can hope that, at worst, it consists of a countable collection of points, open intervals, half-closed intervals, and closed intervals (this is not a general property of subsets of the interval, cf the Cantor set, but it feels very likely that it will apply here).

In that case, these is a continuous surjection from to , mapping each open interval to one of the points or intervals ("folding" over the ends when mapping to those with closed end-points).

Then is the continuous surjection we are looking for.

Note: I'm thinking now that might not be connected, but this would not be a problem as long as it has a countable number of connected components.

[-]DacynΩ110

I am going to take some license with your question because I think you are asking the wrong question. Arbitrary topological spaces and abstract continuity are rarely the right notions in real-world situations. Rather, uniform continuity on bounded sets usually better corresponds to the intuitive notion of "a small change in input produces a small change in output".

Thus, suppose that is a complete separable metric space and that is uniformly continuous on bounded sets. Then we can show that there exists a function which is uniformly continuous on bounded sets but not a fiber of (i.e. there is no such that for all ). Indeed, consider two cases:

1. is uniformly discrete. Then every map from to is uniformly continuous, so we get a contradiction from cardinality considerations.

2. is not uniformly discrete. Then for each n, since f is uniformly continuous on it has a modulus of continuity on this set, i.e. a continuous increasing function such that for all . Since is not uniformly discrete, there is a function such that for infinitely many , there exist such that . (To construct it, take pairs with , extract a subsequence that behaves geometrically nicely, and then find a function such that for all in the subsequence.) Clearly, cannot be a fiber of .

can be a fiber of , since for each , and could be distance greater than from the basepoint.

Example: let , with for and . Let be the basepoint (so that is the point you were calling ""). Let , , , and .

I also don't see how to even construct the function , or, relatedly, what you mean by "geometrically nicely", but I guess it doesn't matter.

Also, I'm not convinced that metric spaces with uniform continuity on bounded subsets is a better framework than topological spaces with continuity.

This is intended as a reply to David Simmons's comment, which for some reason I can't reply to directly.

In the new version of your proof, how do we know isn't too close to for some ? And how do we know that is uniformly continuous on bounded subsets?

About continuity versus uniform continuity on bounded sets:

It seems to me that your point 1 is just a pithier version of your point 4, and that these points support paying attention to uniform continuity, rather than uniform continuity restricted to bounded sets. This version of the problem seems like it would be a less messy version of the "fixed modulus of continuity" version of the problem you mentioned (which I did not understand your solution to, but will look again later).

I'm not sure what you're getting at about singularities in point 3. I wouldn't have asked why you were considering uniform continuity on bounded sets instead of uniform continuity away from singularities (in fact, I don't know what that means). I would ask, though, why uniform continuity on bounded sets instead of uniform continuity on compact sets? As you point out, the latter is the same as continuity.

Your point 2 is completely wrong, and in fact this is the primary reason I was convinced that continuity is a better thing to pay attention to than uniform continuity on bounded sets. The type of object you are describing is an effective Polish space that remembers its metric. Typically, descriptive set theorists forget the metric, and the isomorphisms between Polish spaces are homeomorphisms (and the isomorphisms between effective Polish spaces are computable homeomorphisms). Changing the metric in a computably homeomorphic way does not change what can be done when points are represented as descending chains of basic open sets with singleton intersection. So the thing you described was really topological rather than metric in nature, even though it involves introducing a metric in the setup. I am not aware of any notions of computability in metric spaces in which the metric matters in the way you are suggesting. It is not true that uniform continuity gives you an algorithm for computing the function. As a counterexample, let be an uncomputable number, and let be given by for every . is clearly uniformly continuous, but not computable. It is also not true that uniform continuity is necessary in order for the function to be computable. For instance, is computable on . Of course, is not complete, but for another example, consider an effective infinite-dimensional Hilbert space, and let be an effective orthonormal basis. Let . This sequence of functions is computable, they have disjoint support, and for any point, a sufficiently small neighborhood around it will be disjoint from the supports of all but at most one of these functions. Thus is computable, but of course it is not uniformly continuous on the unit ball, which is bounded. However, it is true that every computable function is continuous, and conversely, every continuous function is computable with respect to some oracle. Of course, we really want computability, not computability with respect to some oracle, but this still seems to show that continuity is at least a good metaphor for computability, whereas uniform continuity on bounded sets doesn't seem so to me.

Of course, all this about continuity as a metaphor for computability makes the most sense in the context of Polish spaces, and we can only talk about actual computability in the context of effective Polish spaces. Scott's problem involves a space and the exponential . If is a locally compact Polish space, then is also Polish. I think that this might be necessary (that is, if and are Polish, then is locally compact), although I'm not sure. If so, and if your proof is correct, it seems plausible that your proof could be adapted to show that there is no locally compact Polish space with the property that Scott was looking for, and that would show that there is no solution to the problem in which and are both Polish spaces, and hence no computable solution, if computability is formalized as in effective descriptive set theory.

[-]DacynΩ120

I don't know why my comment doesn't have a reply button. Maybe it is related to the fact that my comment shows up as "deleted" when I am not logged in.

Sorry, I seem to be getting a little lazy with these proofs. Hopefully I haven't missed anything this time.

New proof: ... We can extract a subsequence such that if and , then for all , and for all and , either (A) and or (B) and . By extracting a further subsequence we can assume that which of (A) or (B) holds depends only on and not on . By swapping and if necessary we can assume that case (A) always holds.

Lemma: For each there is at most one such that .

Proof: Suppose and , with . Then , a contradiction.

It follows that by extracting a further subsequence we can assume that for all .

Now let be an increasing uniformly continuous function such that and for all . Finally, let . Then for all we have . On the other hand, for all we have , for we have , and for we have . Thus . Clearly, cannot be a fiber of . Moreover, since the functions and are both uniformly continuous, so is .

Regarding your responses to my points:

I guess I don't disagree with what you write regarding my points 1 and 4.

It seems to be harder than expected to explain my intuitions regarding singularities in point 3. Basically, I think the reasons that abstract continuity came to be considered standard are mostly the fact that in concrete applications you have to worry about singularities, and this makes uniform continuity a little more technically annoying. But in the kind of problem we are considering here, it seems that continuity is really more annoying to deal with than uniform continuity, with little added benefit. I guess it also depends on what kinds of functions you expect to actually come up, which is a heuristic judgement. Anyway it might not be productive to continue this line of reasoning further as maybe our disagreements just come down to intuitions.

Regarding my point 2, I wasn't very clear when I said that uniform continuity gives you an algorithm, what I meant was that if you have an algorithm for computing the images of points in the dense sequence and for computing the modulus of continuity function, then uniform continuity gives you an algorithm. The function would be the kind of thing I would handle with uniform continuity away from singularities (to fix a definition for this, let us say that you are uniformly continuous away from singularities if you are uniformly continuous on sets of the form , where is some set of singularities).

In your definition of , I think you mean to write max instead of min. But I see your point, though the example seems a little pathological to me.

Anyway, it seems that you agree that it makes sense to restrict to Polish spaces based on computability considerations, which is part of what I was trying to say in 2.

If you have a locally compact Polish space, then you can find a metric with respect to which the space is proper (i.e. bounded subsets are compact): let , where is compact. With respect to this metric, continuity is the same as uniform continuity on bounded sets, so my proof should work then.

Proposition: Let be a Polish space that is not locally compact. Then (with the compact-open topology) is not first countable.

Proof: Suppose otherwise. Then the function has a countable neighborhood basis of sets of the form where is compact and is open. Since is not locally compact, there exists a point such that is not compact for any . For each , we can choose . Let , and note that is compact. Then is a neighborhood of . But then for some . This contradicts the fact that , since we can find a bump function which is on but at .

It does still seem to me that most of the useful intuition comes from point 4 of my previous comment, though.

It appears that comments from new users are collapsed by default, and cannot be replied to without a "Like". These seem like bad features.

Your proof that there's no uniformly continuous on bounded sets function admitting all uniformly continuous on bounded sets functions as fibers looks correct now. It also looks like it can be easily adapted to show that there is no uniformly continuous admitting all uniformly continuous functions as fibers. Come to think of it, your proof works for arbitrary metric spaces , not just complete separable metric spaces, though those are nicer.

I see what you mean now about uniform continuity giving you an algorithm, but I still don't think that's specific to uniform continuity in an important way. After all, if you have an algorithm for computing images of points in the countable dense set, and a computable "local modulus of continuity" in the sense of a computable function with and , then is computable, and this does not require to be uniformly continuous. Although I suppose you could object that this is a bit circular, in that I'm assuming the "local modulus of continuity" is computable only in the standard sense, which does not require uniform continuity.

I'm not sure why you would allow singularities at some points (presumably a uniformly discrete set, or something like that) while still insisting on uniform continuity elsewhere. It still seems to me that the arguments for uniform continuity rather than continuity all point to wanting uniform continuity entirely, rather than some sense of local uniform continuity in most places.

Thanks for pointing out the error in my definition of ; I've fixed it.

In your argument that locally compact Polish spaces can be given metrics with respect to which they are proper, it isn't true that is necessarily a proper metric. For instance, consider a countably infinite set with for . This is a locally compact Polish space, but for every , so , and the space is not proper.

Your last proposition looks correct (though with a typo: last in the proof should be ). However, if is not locally compact, then the compact-open topology isn't necessarily the right topology to consider on . We want a topology making into an exponential object, and it isn't clear that such a topology even exists, or that it is the compact-open topology if it does exist (though it must be a refinement of the compact-open topology if it does exist). Maybe asking about non-locally compact Polish spaces with a Polish exponential space is a kind of weird question, though, and if we're even considering non-locally compact Polish spaces, we should turn to the version of the question where we just want a continuous function admitting all continuous functions as fibers.

[-]DacynΩ120

I will have to think more about the issue of continuity vs uniform continuity. I suppose my last remaining argument would be the fact that Bishop--Bridges' classic book on constructive analysis uses uniform continuity on bounded sets rather than continuity, which suggests that it is probably better for constructive analysis at least. But maybe they did not analyze the issue carefully enough, or maybe the relevant issues here are for some reason different.

To fix the argument that every locally compact Polish space admits a proper metric, let be as before and let if and if . Next, let , where is a countable dense sequence. Then is continuous and everywhere finite. Moreover, if , then and thus is compact. It follows that the metric is proper.

Anyway I have fixed the typo in my previous post.

Hm, perhaps I should figure out what the significance of uniform continuity on bounded sets is in constructive analysis before dismissing it, even though I don't see the appeal myself, since constructive analysis is not a field I know much about, but could potentially be relevant here.

is the reciprocal of what it was before, but yes, this looks good. I am happy with this proof.