The idea here is to simulate a situation where we want the model to use the context information regardless of its own knowledge. Indeed, the contextual knowledge is not the golden world knowledge in this dataset, but it simulates any scenario where there is a new knowledge we want the model to use regardless of its own knowledge.
In this paper, we present a novel method of understanding embeddings by transforming embedding space into a comprehensible conceptual space. We present an algorithm for deriving a conceptual space with dynamic on-demand granularity. We also show a method for transferring any vector in the original incomprehensible space to an understandable vector in the conceptual space. We combine human tests with cross-model tests to show that the conceptualized vectors indeed represent the semantics of the original vectors. We also show the use of our method for various tasks, including comparing the semantics of alternative models.
The method works as follows:
... (read 295 more words →)We have thus defined a meta-algorithm CES (Conceptualizing Embedding Spaces) that, for any given embedding
This example is from the DisentQA dataset. This dataset constructs Counterfactual contextual examples.
We aim to use this dataset as if the contextual information is a piece of new information that we want the model to use instead of its own knowledge.
I agree that the fact that the "23 chromosomes" is kept in the examples can be a bit misleading. But I believe that reading the whole example makes one understand that the contextual answer is 2.
Adding a few more examples from this dataset:
question: actor who plays justin in home and away?\ncontext: Michael Crawford ( born 21 October 1975 ) is an Australian stage , television and film actor , best known for his appearances... (read more)