It seems useful to consider agents that reason in terms of an unobservable ontology, and may have uncertainty over what this ontology is. In particular, in Dialogic RL, the user's preferences are probably defined w.r.t. an ontology that is unobservable by the AI (and probably unobservable by the user too) which the AI has to learn (and the user is probably uncertain about emself). However, onotlogies are more naturally thought of as objects in a category than as elements in a set. The formalization of an "ontology" should probably be a POMDP or a suitable Bayesian network. A POMDP involves an arbitrary set of states, so it's not an element in a set, and the class of POMDPs can be naturally made into a category. Therefore, there is need for defining the notion of a probability measure over a category. Of course we can avoid this by enumerating the states, considering the set of all possible POMDPs w.r.t. this enumeration and then requiring the probability measure to be invariant w.r.t. state relabeling. However, the category theoretic point of view seems more natural, so it might be worth fleshing out.

Ordinary probably measures are defined on measurable spaces. So, first we need to define the analogue of "measurable structure" (-algebra) for categories. Fix a category . Denote the category of measurable spaces. A measurable structure on is then specified by providing a Grothendick fibration and an equivalence . Here, stands for the essential fiber of over the one point space . The intended interpretation of is, the category of families of objects in indexed by measurable spaces. The functor is supposed to extract the base (index space) of the family. We impose the following conditions on and :

Given , and , we denote the corresponding base change by ( and is canonically isomorphic to ).

  • Consider and . Consider also a point . We can think of as a morphism . This allows us considering the base changes and (the "fibers" of at and at respectively) where . Applying the universal property of to and , we get morphisms . We now require that, if for any , then (morphisms between families that are pointwise equal are just equal).

  • Consider and . Suppose that (i) is an isomorphism and (ii) for any , is an isomorphism. Then, is an isomorphism (families with a common base that are pointwise isomorphic are just isomorphic).

I'm not entirely sure how sufficient or necessary these conditions are for proving useful results, but they seem to me natural at first glance. Note that this definition can be regarded as motivated by the Yoneda lemma: a measurable space is defined by the measurable mappings to from other measurable spaces, so a "measurable category" should be defined by the measurable "mappings" to it from measurable spaces, and is precisely the category of such measurable "mappings". Compare this with definition of geometric stacks(fn1).

Next, we define probability measures. Specifically, for any "measurable category" (a category equipped with structure as above), we construct the category of "probability measures on ". First, we define the auxiliary category . An object in is a pair where is an object in and is a probability measure on . We interpret this as sampling from and then taking (using , the latter can be considered to be an object in ). We define the morphisms from to as those morphisms for which (the notation stands for pushforward). Given , we call it a "quasi-isomorphism" when, for any , is an isomorphism. Claim: quasi-isomorphisms admit a calculus of right fractions(fn2). We now define as the localization of by quasi-isomorphisms.

(fn1) Maybe the analogy with stacks should be made more formal? Not sure, stacks are motivated by topology and measurable spaces are not topological...

(fn2) This should clearly be right, and this is right for natural examples, but I haven't written down the proof. If it turns out to be false it would mean that my conditions on are too weak.

Vanessa Kosoy's Shortform

by Vanessa Kosoy 18th Oct 201957 comments
Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.