Sorry, I think I might have a superficial understanding of encoders and embeddings. Would you be able to try pointing out for me how decomposition is performed in that case (or point me toward a favorite reading on the subject)? When I think of feeding a sentence into an encoder, I can think of multiple ways in which some compositional structure might be inferred.

I'm drawing up a proof of concept with seq2seq learners right now, but my hypothesis is that they will be inadequate decomposers suitable only for benchmarking a baseline.

Reply

3George3d65y

I was asking why because I wanted to understand what you mean by "decomposition". Defines many things. Usually the goal is feature extraction (think Bert) or reducing the size of a representation (think autoencoders or simpler , PCA) You need to narrow down your definition, I think, to get a meaningful answers.

2abramdemski5y

I think Quinn means factored cognition, which is quite different from autoencoders/embeddings/PCA.

1Quinn5y

Thank you Abram. Yes, factored cognition is more what I had in mind. However, I think it's possible to speak of decomposition generally enough to say that PCA/SVD is a decomposer, albeit an incredibly parochial one that's not very useful to factored cognition. Like, my read of IDA is that the distillation step is proposing a class of algorithms, and we may find that SVD was a member of that class all along.

Rendering 0/4 comments, sorted by

top scoring

(show more) Click to highlight new comments since: Today at 11:07 AM

Moderation Log

LESSWRONG
LW

LESSWRONG
LW

8

[ Question ]

Have general decomposers been formalized?

8

Ω 4

8

Ω 4

1 Answers sorted by
top scoring

Jun 27, 2020

8

[ Question ]

Have general decomposers been formalized?

8

Ω 4

8

Ω 4

1 Answers sorted by top scoring

Jun 27, 2020

1 Answers sorted by
top scoring