Using Finite Factored Sets for Causal Representation Learning?

11th Jan 2023

3Matthias G. Mayer

1David Reber

2Matthias G. Mayer

New Answer

New Comment

1 Answers sorted by

I'm working on the FFS framework in general. I'm currently writing up decidability of finite temporal inference. After this I will probably start working on efficient finite temporal inference which is what you're referencing if I understood correctly.

I'm also working on extending the framework to the infinite setting and am almost finished except for conditional orthogonality for uncountable sets.

I quite like the name Logical Time Theory, under which I will probably publish those results in a month or so.

I'm also working on extending the framework to the infinite setting and am almost finished except for conditional orthogonality for uncountable sets.

Hmm, what would be the intuition/application behind the uncountable setting? Like, when would one want that (I don't mind if it's niche, I'm just struggling to come up with anything)?

21y

A direct application would need that you have an uncountable variable. You might want to do this if you have enough evidence to say this confidently. As a simple example imagine a real-valued graph where all your data points lie almost on the identity diagonal. You might then want to infer a variable which is the identity.
As a more general application, we want to model infinities because the world is probably infinite in some aspects. We then want a theorem that tells us, that even if the underlying model is infinite, if you have enough data points then you are close enough, like with the Strong law of Large numbers, for example.

One of the primary motivations of Finite Factored Sets (shorthand: FFS) that initially caught my eye was, to quote Scott:

And this becomes apparent with the toy model Magdalena analyzes at the end of her distillation post: taking variables as primitives (as in the Pearl framework) means you have to make arguments about 'whether deterministic collapse occurs', rather than variables arising naturally as in finite factored sets.

So:

EDIT: It may help to know that my motivation is "Can we apply a FFS algorithm for causal representation learning to learn objects (and physics) from a video? Or (more directly for alignment), to identify latent concepts embedded in a LLM?"