Sometimes you also need to be careful not to specify too much background for the question. Suppose you want to evaluate a controversial hypothesis by asking a simple, non-controversial factual question. Then what you want to do is to just ask the factual question, without connecting it to the controversial hypothesis. Otherwise people will end up debating the controversial hypothesis and never answer the actual question you were interested in.
This not only happened to me recently, but also expanded to a flamewar elsewhere. I should've just asked "what gender ratios have you observed".
Obviously this is more of an issue in online forum-type conversations, but I've also observed in one-to-one conversations.
Did any useful communication happen on that second thread? It looks like an epic identity politics circle jerk.
Did any useful communication happen on that second thread?
No, not really.
It wasn't quite as bad as the one time a message board decided I was a fundamentalist nut because I had the audacity to suggest that religion may also have benefits (even if those were possibly outweighed by the harms). But I did get a similar "it doesn't really matter what you say, we've already decided to interpret it in the worst possible way regardless" vibe.
It wasn't quite as bad as the one time a message board decided I was a fundamentalist nut because I had the audacity to suggest that religion may also have benefits (even if those were possibly outweighed by the harms).
Related article from the sequences: Policy Debates Should Not Appear One-Sided.
True. But in the party-guests example, at least, having any kind of answer, biased or not, is better than an "I don't know".
Yeah, it is usually better to have an answer, even if it's slightly skewed, rather than literally no idea. One technique you could use is to provide two anchors, one way too low and the other way too high, so the anchoring effect is limited. For the party example (unless you're a frat house), you could use a low of 8 people and a high of 75.
I haven't read any studies on anchoring effects when two anchors are given, like this. Anyone got a link?
Depends. A convoluted counterexample:
Alice has to buy food for a party while Bob is sending invitations. Alice initially expects 10 guests to arrive and plans to buy corresponding amount of food. For simplicity let's suppose that having 1-person worth of food more than needed bears the same cost as having the same amount less than needed, and Alice's prior probability distribution is symmetric around 10, so she intends to buy 10 units of food.
But before going to the shop, she asks Bob how many people are coming. Bob is reluctant to answer, so Alice offers an arbitrary threshold 40. Actually only 15 guests are going to attend, but Bob doesn't know this. His prior expectations were more or less the same as Alice's, but by power of anchoring, he gives the answer "more than 40" with p=0.3. This answer causes Alice to buy 45 (the lowest round number greater than 40) units of food. If Bob answers "not more than 40", Alice will stick with her previous guess. Which means the cost is 0.3 30 + 0.7 5 = 12.5.
Without asking, Alice would buy 10 units of food and the cost of error would be only 5.
I dont really think it would work like that. My understanding of anchoring is that they think 40 and then search for the nearest reasonable answer and whether that's above or below the number correlates to their initial internal beliefs about the value in question. That is to say, if he originally thought it was ten, he would think 40, and then consider (subconsciously) all numbers between 40 and zero in decreasing order. I expect he would stop no earlier than 20 (or maybe as low as 15 :D).
I assumed he was unwilling to tell an exact number before, so he would probably not do it afterwards. He will only tell whether it is higher or lower than 40, and I suppose anchoring will move the median of the probability distribution towards the anchor, thus increasing the probability of answering "more than 40" in this case.
I think about making a poll here to test the hypothesis, only not sure how to do that.
I disagree. I suppose it may slightly increase the probability of the "more than 40" answer, but his prior expectations have to play some role. Since there would be no other real factors, the effect of his prior expectation would in my opinion shift his answer downwards, yielding the "less than 40" answer. Anchoring pulls you towards the suggested value, but I'm not sure in this case it would pull it ABOVE the suggested value, especially given the difference between his original beliefs and the anchor.
OTOH, "what kind of car is it" is a more revealing question, in that it will yield more information about what metadata matters to your interlocutor. (It's what I'd call a more open question.)
Conversely "what color is it" conveys information about what metadata matters to you. If you think about it, it's strange for a question to convey information; more often when you ask a question you want to collect information.
Often you'll want to start an information-seeking conversation with open questions, the ideal being "context free questions" - questions that are likely to elicit an answer irrespective of the situation you ask them in. Then as the answers lead you to generate hypotheses you'll want to ask confirming questions to probe for false assumptions.
Garbled or missing metadata can cost time and cause fights
Citation needed. (Or an example or three.)
Speaking as someone who asks questions for a living, some of your advice in this post strikes me as dubious. I kind of agree with your caveats about definitions, but they seem weakly connected to the general notion of "metadata".
Citation needed. (Or an example or three.)
(Heavily fictionalized but not made up examples follow):
Interlocutor: Hey, have you read Exemplar?
Me: I don't think I'm familiar with it.
Interlocutor: Are you sure? We have a copy around here somewhere. It's by X. Ample and -
Me: Oh, is it blue?
Interlocutor: ...How should I know?
Me: I think I remember reading a blue book by somebody with an X initial... did it have a character who kept saying "prithee" at random times?
Interlocutor: I told you, the book is called Exemplar, it's by X. Ample, have you read it or not?
(And then I have to go physically locate the blue book I'm thinking of to see if it is this one.)
Friend Who Used To Be Willing To Talk Religion With Me: Deploys extended metaphor using the word "authority".
Me: continues metaphor, leaning on connotations of "authority"
FWUTBWTTRWM: No! Now I'm offended! Rar!
Me: Bwuh? But I just - you said -
FWUTBWTTRWM: No, that's not what we're talking about! I don't mean these six other things I have discussed before that also use the word authority, they're different! Rar!
I can see how the first example relates to your post. I'm not seeing (yet) how the concept "metadata" applies to the second example. It feels more like "inferential distance costs time and causes fights".
The first example does raise interesting questions of ontology, with at least three different concepts in play (roughly "book", "copy" and "story" with corresponding metadata categories of "bibliographic entry", "physical characteristics" and "narrative structure"). Any particular book you read is going to leave recollections from each of these categories in your mind and it's not so surprising that you don't always control how much you remember from which categories. (I hear that writing a review of each book you read helps pin down some of these memories; I don't typically do that.)
I can see how being explicit about those can be helpful. The choice of terminology might be unfortunate: "metadata" is likely to make sense primarily to people who have some training in computer programming, and that training has already made them (if they're at all competent) more explicitly aware of these matters of ontology. So in terms of delivering useful advice the post as written might be coals to Newcastle. Maybe you could start off with something like "Programmers have a concept of metadata, i.e. information used to tag other information..." to help a more general audience bridge the gap.
For your own practical purposes, though, in the first example it sounds as if you're being as helpful as you can and your interlocutor in contrast is being impatient - a matter of attitude rather than a matter of metadata.
Get clear about motives.
For example, when my husband asks me "Do you want to go out tonight?" I generally reply "Why do you ask?" Sometimes he is motivated by a desire to get out of the house; sometimes he is motivated by a desire to do laundry (which means he can't leave the house for a while).
Similarly, I try to preface questions with a brief explanation of why I care. For example, I prefer "I came across this stuff while cleaning the basement and was going to throw it out, but I figured I'd check with you first: do you want to keep any of it?" to "Do you want to keep any of this stuff?"
You make some good points here, Alicorn.
One thing to watch out for: When you find out that your interlocutor has interpreted your question differently than you intended it may be polite/friendly to let her finish speaking before following up with the more specific thing you wanted to know. People (often) enjoy open conversation more than specific interrogation.
I seem to have this problem with my mother-in-law, we don't understand each other very well. A recent conversation went like this.
MIL: So, what sort of girl is this babysitter you've hired?
Me: Oh, she's a lovely sweet country girl, a bit shy. This one time she was... [want to launch into a funny story]
MIL: [interrupts] No, no, I mean - is she a student or does she work somewhere?
Me: [annoyed that I didn't get to tell my story] She's a student.
Agreed. That seems to exemplify the difference between social interaction for its own sake, and social interaction for the sole purpose of gathering certain information or getting something done. I think the point in my life when I became a good listener (or at least a better listener than before!) is when I started seeing people as versions of me rather than as confusing-but-potentially-useful furniture. And I like to have a chance to tell stories, so I give other people that latitude too, and the stories are usually more interesting than whatever fact I might have specifically wanted to learn.
I think the point in my life when I became a good listener (or at least a better listener than before!) is when I started seeing people as versions of me rather than as confusing-but-potentially-useful furniture.
Upvoted for helpfulness.
Interesting, I've had the opposite experience.
I used to view other people as versions of me, and projected my opinions and thoughts into them. I would expect people to understand what I was saying and agree that it was reasonable. All deviations were bad. This didn't work particularly well.
I find viewing them as people with equal moral weight, but different personalities and wants works much better.
Very interesting post. I do think some of your examples are more realistic than others. The "how many people are coming to the party?" example is the most plausible: you want a direct, numerical answer so you can build your plans on it, and a roundabout answer is frustrating because it prevents you from executing any plans.
The "what kind of car?" and "what kind of house?" questions are, I think, naturally more open-ended. In most of the contexts where I would ask those questions, it would be to start a conversation, or to signal interest in the other person's life. (Maybe not when I was eleven, considering I lacked basic social skills back then.) Unless I needed the purely factual information of car colour to make a decision, I wouldn't want to restrict those questions. Even if I want to decide what kind of car to buy, I would probably learn more from letting the other person, who almost certainly knows more about cars than I do, decide what they think is pertinent. Likewise for the house-building; anyone who is building a house of their own probably has a very interesting story about what kind it is and why. And in the process, their story will reveal something about their personality, so that I can model them better, which will allow me to interact with them more efficiently in future because I'll know how they see things and how they communicate. (This is assuming I have time to invest in a conversation, but I consider it a quite important investment, as well as being pleasurable for its own sake.)
However, your point holds very true for conversations that are fundamentally about ideas or concepts and not socializing/personal enjoyment. If you're going to discussion politics or quantum physics, you definitely want information to be transmitted clearly and efficiently, and you want your keywords to mean the same thing to the other person as they do to you.
Sometimes when I ask people how things are going with them, I don't want a lot of technical detail, I want to know something about whether they're happy or not with how things are working out.
More generally, (and this might be worth another post) people are pretty inflexible about how much detail they give, and have trouble going to a more detailed style or a more summarized style than their usual.
More generally, (and this might be worth another post) people are pretty inflexible about how much detail they give, and have trouble going to a more detailed style or a more summarized style than their usual.
I naturally give way more detail (especially qualifiers) than most people want in most situations. I can correct it somewhat in email (by revising; I can't omit unneeded detail without writing it out to see which parts are unneeded) although it takes much longer than just giving the detail. In a realtime conversation I'm not sure, although it's probably less bad because of the tighter feedback loop.
It sounds like you're saying this is a general phenomenon; do you have a cite, or is this based on personal experience?
Interestingly, I have to force myself to write out descriptive details when writing a narrative. Which means it's the kind of information, not just the amount. So maybe it's more linked to this article than you seem to be thinking?
Personal experience. I tend to like hearing less detail than people want to give, and it's difficult to get them to summarize.
People don't generally ask me to change my detail level-- I don't know whether I'm in a sweet spot, the tolerable range, or (as I strongly suspect) most people are much more inhibited about asking for a different detail level than I am.
If you ever figure out any tricks to get others to change, I'd be interested - I run in to the same problem :)
Conversely, I do get a fair number of requests to change my own detail level: I tend to strongly encourage people to ask for elaboration as needed, so I do get a fair number of "requests for additional detail". And occasionally I'll have someone cut me off for having answered a question they didn't intend (i.e. "I meant, what color is it?"). I've found demonstrating positive/happy body language in these situations, and verbally reinforcing that I like this, has made people do that more often.
It may also be a specific, rather than a general inhibition: if someone responded to me in a way that made me believe they wanted less detail than I was providing, I would not be inclined to ask them for more details about their answers even if I wanted more information than that in general.
I've been reflecting on the suggestion to think about "what kind of answer you're looking for" quite a bit recently, not in terms of conversation with others (although it is relevant to my difficulties with prompting LLMs) but in terms of framing problems and self-directed questions.
Morendil's comment about "open questions" made me think about this a bit more:
The concept of "open" and "closed" questions is something I saw a lot of in customer service - you ask broad, open questions like "How can I assist you?" at the start of the call, then start using closed questions to collect specific follow-up data like "what model of phone do you have?"
One of the other things you learn is how to re-frame questions when the other person has no clue what you're talking about: "Okay, is it a flip phone? Okay, then there should be a model number on the back, it will be three digits. Thank you!"
I sadly don't have a training manual available to me, but it occurs to me that it would be a fairly obvious source of useful information on this topic :)
I have these problems a lot, and have tried those solutions, but they rarely seem to work unless the person you're talking to also know of them. And they get cumbersome quickly when you have to do this kind of expansion for more than 2 or 3 terms in a conversation. Non-rationalists tend to find my habit of asking for definitions and clarifications annoying and tell me to stop, sometimes angrily. Holding of on proposing solutions for now.
The Self-Explanatory Magical Amulet
Aren't they all?
The super-powerful weapon...which cannot be used directly against the main antagonist...or is so evil it can't be used at all.
The artifact that gives only a subtle, unmeasurable boost to the hero's abilities...because the magic was inside him or her all along!
The magic whistle that summons help, only to be used in case of an emergency requiring its use...one and only one of which will arise.
The item presented by the old one to the hero who has a humble upbringing but newly discovered noble origins...which always casts insert backstory.
The useful magic item necessary to get the hero through a later challenge...which, when carried by a non-central character, always casts have item bearing non-central character carrying it get fatally wounded, but not so badly he or she dies immediately, and say, I'm dying, please take my item,' to which the hero says 'No non-central character, you can't die!'
The item that actually would kill the hero if used...which always casts insert villain arrogance, usually gloating and/or explaining the evil plan, providing time to avert the hero's death.
The powerful intrinsically evil artifact used by the villain...which casts karmic justice, turning against the villain, even though the hero was not strong enough (or too unwilling to kill) to do so (or kill the villain directly).
Once when I was probably eleven-ish, I asked a friend of my family who had just gotten a new car, "What kind of car is it?" He began to tell me the make and model and the interesting features of this particular vehicle.
I interrupted him, and said, "I meant, what color is it?"
This is just a mildly cute story about how little I knew or cared about cars at age eleven-ish, but it uncovers a communication issue that applies to people who are not eleven-ish anymore. I should have just asked in the first place what color the car was, since that was what I wanted to know. Asking what kind it was allowed a misunderstanding to creep into the interaction, since "kind" doesn't have a fixed meaning as regards cars and my interlocutor attached his own understanding of the question when he interpreted it. I didn't correctly pin down the metadata of my question, so he didn't know what kind of answer I was looking for.
Garbled or missing metadata can cost time and cause fights, so I have developed a number of techniques to mitigate or eliminate it, both incoming and outgoing. They're pretty simple to apply, and bringing them to bear early is very instrumentally useful both for social and informational reasons.
Ask yourself what kind of answer you're looking for, then ask a question that calls for that kind of answer, or give an example of what would constitute a good answer to the person you're asking. Examples of extensions/replacements for the question "What kind of house are you going to build?"
"Will it be, like, a cute suburban place with azaleas out front?"
"I bet you're going with a brick facade, am I right?"
"What square footage are you looking at, ballpark?"
For numerical inquiries that your interlocutor cannot seem to answer, pick one or more threshold numbers (surprisingly often, people who have "no idea" how to answer a numerical question can tell you whether the answer is greater or lesser than an arbitrary quantity). Explain why you need the information. Or ask for a related more-concrete detail that will give you an adequate estimate. Examples of extensions/replacements for the question "How many people are coming to the party?"
"Is it more, or fewer, than 40 guests?"
"I'm just trying to figure out how many boxes of plastic forks to buy - they come in 50ct boxes, and it'd be better to have extra than to fall short if we have to guess."
"How many invites did you send out, and how many RSVPs did you get back, and what did they say?"
Make sure the definitions of key words are agreed upon so you don't miss connotations or implications that are eluding one or both of the persons in an exchange. (You may need to agree to use a silly or even incoherent definition for something to forestall exasperating argument, especially if you are talking to people who believe in things like libertarian free will. Then you can give your own concepts their own labels for the duration of the conversation.) Examples of extension/replacements for the question, "If a tree falls in the forest and no one hears it, does it make a sound?"
"What are you using "sound" to mean here...? Okay, I've been using it differently, but we'll go with that. I'm talking about this other thing which I'll call an alberzle..."
"I've been thinking about kirrenberry trees all day, sorry - they're a fictional plant that doesn't make noise even when people are listening - but I'm guessing that's not what you had in mind when you said tree."
"That's the sixth time you've brought up squirrels - oh, my bad. I should've specified when I asked, I meant nothing with ears is around to hear it, I wasn't trying to specify humans."
Keep track of what kind of metadata your frequent associates natively use, so you can jump immediately to the correct information. This will also let you work backwards, if someone's going on a chain of inferences you can't follow easily ("What? I'm not talking about X at all. Are you thinking of X because it shares a national origin with my topic Y?"). Examples of extensions/replacements for the question "Have you read this book?"
"You would have had my copy - it was blue, with a beat-up corner and the library jacket on?"
"It's The Placeholder by X. Ample. Ample also wrote Exemplar, remember?"
"The main character finds a magical amulet called The Self-Explanatory Magical Amulet and he - the amulet - talks about himself in a faux-medieval dialect the whole book."
I'm sure this isn't an exhaustive list of workarounds for metadata problems. What am I missing?