LESSWRONG
LW

[ Question ]

Have general decomposers been formalized?

1 min read27th Jun 20201 answer 4 comments

8

Ω 4

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

Hi, I'm working on a response to ML projects on IDA focusing on a specific decomposer, and I don't know if someone's formalized what a decomposer is in the general case.

Intuitively, a system is a decomposer if it can take a thing and break it down into sub-things with a specific vision about how the sub-things recombine.

Have general decomposers been formalized?

New Answer

New Comment

1 Answers sorted by
top scoring

Jun 27, 2020

10

Why is the literature into reversible encoders/autoencoders/embedding generators not relevant for your specific usecase ?

Give an answer to that it might be easier to recommend stuff.

Sorry, I think I might have a superficial understanding of encoders and embeddings. Would you be able to try pointing out for me how decomposition is performed in that case (or point me toward a favorite reading on the subject)? When I think of feeding a sentence into an encoder, I can think of multiple ways in which some compositional structure might be inferred.

I'm drawing up a proof of concept with seq2seq learners right now, but my hypothesis is that they will be inadequate decomposers suitable only for benchmarking a baseline.

3George3d64y

I was asking why because I wanted to understand what you mean by "decomposition". Defines many things. Usually the goal is feature extraction (think Bert) or reducing the size of a representation (think autoencoders or simpler , PCA) You need to narrow down your definition, I think, to get a meaningful answers.

2abramdemski4y

I think Quinn means factored cognition, which is quite different from autoencoders/embeddings/PCA.

1Quinn4y

Thank you Abram. Yes, factored cognition is more what I had in mind. However, I think it's possible to speak of decomposition generally enough to say that PCA/SVD is a decomposer, albeit an incredibly parochial one that's not very useful to factored cognition. Like, my read of IDA is that the distillation step is proposing a class of algorithms, and we may find that SVD was a member of that class all along.