LESSWRONG
LW

AI
Frontpage

8

[ Question ]

Have general decomposers been formalized?

by Quinn
27th Jun 2020
AI Alignment Forum
1 min read
A
1
5

8

Ω 4

AI
Frontpage

8

Ω 4

Have general decomposers been formalized?
1George3d6
1Quinn
3George3d6
2abramdemski
1Quinn
New Answer
New Comment

1 Answers sorted by
top scoring

George3d6

Jun 27, 2020

10

Why is the literature into reversible encoders/autoencoders/embedding generators not relevant for your specific usecase ?

Give an answer to that it might be easier to recommend stuff.

Add Comment
[-]Quinn5y10

Sorry, I think I might have a superficial understanding of encoders and embeddings. Would you be able to try pointing out for me how decomposition is performed in that case (or point me toward a favorite reading on the subject)? When I think of feeding a sentence into an encoder, I can think of multiple ways in which some compositional structure might be inferred.

I'm drawing up a proof of concept with seq2seq learners right now, but my hypothesis is that they will be inadequate decomposers suitable only for benchmarking a baseline.

Reply
3George3d65y
I was asking why because I wanted to understand what you mean by "decomposition". Defines many things. Usually the goal is feature extraction (think Bert) or reducing the size of a representation (think autoencoders or simpler , PCA) You need to narrow down your definition, I think, to get a meaningful answers.
2abramdemski5y
I think Quinn means factored cognition, which is quite different from autoencoders/embeddings/PCA.
1Quinn5y
Thank you Abram. Yes, factored cognition is more what I had in mind. However, I think it's possible to speak of decomposition generally enough to say that PCA/SVD is a decomposer, albeit an incredibly parochial one that's not very useful to factored cognition. Like, my read of IDA is that the distillation step is proposing a class of algorithms, and we may find that SVD was a member of that class all along.
Rendering 0/4 comments, sorted by
top scoring
(show more)
Click to highlight new comments since: Today at 5:32 PM
Moderation Log
Curated and popular this week
A
1
0

Hi, I'm working on a response to ML projects on IDA focusing on a specific decomposer, and I don't know if someone's formalized what a decomposer is in the general case.

Intuitively, a system is a decomposer if it can take a thing and break it down into sub-things with a specific vision about how the sub-things recombine.