Authors Have a Responsibility to Communicate Clearly
35Kaj_Sotala
16Vladimir_Nesov
32habryka
1iainlim
4Benquo
5plex
33Neel Nanda
17[anonymous]
6AnthonyC
13Jozdien
12mattmacdermott
11Martin Randall
9Aprillion
13Martin Randall
5Aprillion
2Martin Randall
3Jiro
5Martin Randall
2Jiro
10Gunnar_Zarncke
9mattmacdermott
6wonder
4Steven Byrnes
2Gunnar_Zarncke
3DanielFilan
2Gavin Runeblade
2Said Achmiz
2MondSemmel
2Said Achmiz
New Comment
It used to be that I would sometimes read something and interpret it to mean X (sometimes, even if the author expressed it sloppily). Then I would say "I think the author meant X" and get into arguments with people who thought the author meant something different. These arguments would be very frustrating, since no matter how certain I was of my interpretation, short of asking the author there was no way to determine who was right.
At some point I realized that there was no reason to make claims about the author's intent. Instead of saying "I think the author meant X", I could just say "this reads to me as saying X". Now I'm only reporting on how I'm personally interpreting their words, regardless of what they might have meant. That both avoids pointless arguments about what the author really meant, and is more epistemically sensible, since in most cases I don't know that my reading of the words is what the author really intended.
Of course, sometimes I might have reason to believe that I do know the author's intent. For example, if I've spent quite some time discussing X with the author directly, and have a good understanding of how they think about the topic. In those cases I might still make claims of their intent. But generally I've stopped making such claims, which has saved me from plenty of pointless arguments.
The useful thing is the ideas you take away, it's rarely relevant if the author intended them or not.
2In policy contexts (not just political policy, but like, making group decisions of almost any kind), it tends to matter a lot what the author intended, since that's often a natural schelling point for resolution of ambiguities that later gets referred to.
See for example courts trying to interpret what previous courts intended with a judgement, or what a law was intended to do. Same for company decisions to move ahead with a project. In almost any context with stakes, the intention of the author continues to matter (though how much varies from context to context, though my sense is almost always a good amount).
2Yes but it's also not rare for legal scholars to see "legislative intention" as nothing but a convenient fiction that courts employ to retain some semblance of objectivity. In principle, they could drop such language and refer directly to their interpretation of the statutory text, right? The intellectual labor involved is unchanged: each judge considers their community's surrounding norms and common sense assumptions to understand what a law is "really" trying to do. Whether or not this interpretation is ascribed to authorial intention is basically superfluous.
At least some scholars see a meaningful difference between "original intent" and "original public meaning."
This feels very flavoured like A Principled Cartoon Guide to NVC's view on avoiding communication failures!
This seems somewhat backwards to me. Communication is kind of like a tree search of meaning. Everything you say is full of ambiguity and you can spend more words clarifying them by choosing specific nodes of the tree. To dig into this means you need to prioritise which bits are important, you can't make everything precise. I generally disapprove of critiquing people who don't put in the effort to clearly communicate a thing that was clearly unimportant to that point and I view the sloppiness defence as saying that "you are making an implicit argument that this thing is low quality because it has an error. However, the error is in a part of the tree that wasn't very relevant for the arguments. So who cares"
Separately, there is someone who supports their case with reasoning that is invalid and ambiguity defends them from people making this critique. But there, even if you give them the benefit of the doubt, the updated argument does not support the conclusion, so sloppiness is irrelevant - their argument is invalid
Maybe there's a gray zone where the detail seems plausibly relevant to a reader but actually isn't? Critique seems reasonable here.
I don't know which category the Bengio thing falls into - it seems like it makes an invalid argument to support a specific point in its overall thesis? I haven't read it
Regardless of who was "right", this conversation is a springboard for examining the broader conversational move of claiming "the author was right but was writing sloppily."
Without getting into the specifics of this particular example,[1] sometimes this pattern occurs when an author wants to communicate some intuitive conclusion they have reached, but that intuition is the result of interacting such a large amount of literature/research/empirics/self-reflection/personal experience etc. that they can no longer point to any single short and compelling argument for why it's true.
Mindful of norms and razors such as "what can be asserted without evidence can also be dismissed without evidence," they believe saying "I believe X because of intuition, but I can't explain why" would be dismissed by their audience,[2] so they instead try to manufacture some makeshift argument for it because it feels more epistemically virtuous[3] to come up with "proper" scientific evidence. Unfortunately, their argument is often subtly wrong or slightly off-topic, which shouldn't be surprising; after all, it's not the argument that actually caused them to believe their conclusion, but more akin to a post-hoc confabulation.
See also Kaj Sotala's comment here and Ben Pace's summary of Jan Kulveit's comment here.
I think this is an important point, especially when experts are talking to other experts about their respective fields. I once had a client call this "thinking in webs." If you have a conclusion that you reached via a bunch of weak pieces of evidence collected over a bunch of projects and conversations and things you've read all spread out over years, it might or might not be epistemically correct to add those up to a strong opinion. But, there may be literally no verbally compelling way to express the source of that certainty. If you try, you'll have forgotten most of the evidence, and even if you don't, you'll end up with an audience that's bored to tears and (even if they're trying to understand) able to easily explain away each individual piece of evidence.
One canonical example might be Niels Bohr telling Einstein to quit telling God what to do. Not everything Einstein thought was right, but he clearly had and used very strong intuitions, based on thinking about known examples, regarding the nature of physical law. These were difficult to express and (rightly) would not, themselves, be considered scientifically valid evidence for drawing conclusions. IIRC I think that specific retort was in regards to "God does not play dice," which AFAIK most physicists then and since have disagreed with. But, depending on what specifically Einstein meant, and how the universe is actually operationalizing the underlying physics we see as quantum mechanics, it is still entirely possible he was right, but in a different way then he would have expected at the time.
Thanks for the post! I think there are particularly egregious examples where this clearly holds—e.g., I think it's probably good to have a norm that when pushed on something specific that's ambiguous authors say "[This] is what I meant", instead of constantly evading being pinned down.
That said, I think there are many examples of essays where authors just did not foresee a potential interpretation or objection (an easy example is an essay written years before such discussion). Such writing could conflate things because it didn't seem important not to—I think precision to the degree of always forestalling this is ~infeasible. I've talked to people who point at clear examples of this as examples of sloppy writing or authors reneging on this responsibility, when I think that particular ask is far too high.
To be clear, this is to forestall what I think is a potential interpretation of this post, not to imply that's what you meant.
Not sure how to think about this overall. I can come up with examples where it seems like you should assign basically full credit for sloppy or straightforwardly wrong statements.
E.g. suppose Alice claims that BIC only make black pens. Bob says, "I literally have a packet of blue BIC pens in my desk drawer. We will go to my house, open the drawer, and you will see them." They go to Bob's house, and lo, the desk drawer is empty. Turns out the pens are on the kitchen table instead. Clearly it's fine for Bob to say, "All I really meant was that I had blue pens at my house, the point stands."
I think your mention of motte-and-baileys probably points at the right refinement: maybe it's fine to be sloppy if the version you later correct yourself to has the same implications as what you literally said. But if you correct yourself to something easier to defend but that doesn't support your initial conclusion to the same extent, that's bad.
EDIT: another important feature of the pens example is that the statement Bob switched to is uncontroversially true. If on finding the desk drawer empty he instead wanted to switch to, "I left them at work", then probably he should pause and admit a mistake first.
Based on your description of that incident, I expect that when Bob said:
"I literally have a packet of blue BIC pens in my desk drawer. We will go to my house, open the drawer, and you will see them."
that he did not, in fact, really mean:
"I have blue pens at my house"
That is a bizarre interpretation of that set of words. Why did Alice and Bob open the drawer to look for pens if they didn't both interpret his words as indicating that there was, literally, a packet of blue BIC pens in Bob's desk drawer?
So when Bob says:
"All I really meant was that I had blue pens at my house"
then he is making a false statement. Perhaps he is knowingly making a false statement - that's ethically bad. Perhaps he is unable to comprehend the meaning of his own words - that's epistemically bad. Either way it's not fine.
Instead Bob could say a true thing like:
"I was wrong about where the pens were, but right about BIC making blue pens"
Or a shorter true thing like:
"See, BIC makes blue pens".
My score for Bob:
- full credit for knowing whether BIC makes blue pens.
- partial credit for mostly knowing the location of his pens.
- large demerit for being untrustworthy about the meaning of his own words, in a low stakes situation where there was no reason to lie. Get a grip, Bob.
huh? I see the bizarreness in the opposite way - why would you score Bob in such an anti-useful way?
to me, it seems blindingly obvious what Bob was trying to communicate and that Alice would be completely on Bob's side without any untrustworthiness points about any words or anything else between them, not even a hint... they looked into the drawer because that's where Bob remembered leaving them, then they both concluded that the exact statement is wrong and that they should focus on the gist of the problem they are trying to solve - if they both see the blue pens on the table, they both agree about the existence of blue BIC pens and they both agree that either Bob's memory is faulty or that someone else moved the pens, but that it doesn't matter, that the argument Bob should have made was a weaker version of what he actually said when he was too enthusiastic, that a more carefully worded argument would have been warranted but that what he meant was that he can prove the existence of blue pens by showing them to Alice
I believe Bob has a grip, and that Alice does not even need to be that extra charitable, just reasonable / not adversarial. There is no need to paint Bob in any wrong light here, the structure of this story is completely within the bounds of intellectual standards proposed in the OP, I don't see what could possibly be said against Bob here in any honest manner.
If Bob's prediction was written down "for the record" somewhere, that place should be updated by the new definition that both Bob and Alice completely agree about, for the benefit of onlookers - yes, I can agree with that! But in this story, Bob should NOT be accused of anything - not even that he was sloppy - given the everyday-life stakes of this story, the level of precision was entirely appropriate IMHO.
We may have different values, or we may be imagining a different hypothetical, or something else. I'll try to elaborate.
Like you, it seems blindingly obvious to me what Bob was trying to communicate by saying "I literally have a packet of blue BIC pens in my desk drawer". To be explicit, Bob is stating his belief that there is a packet of blue BIC pens in his desk drawer. He doesn't specify his credence, but it's probably somewhere 90-99% depending on other factors.
Bob's words imply other things that have been mentioned, all of which are straightforwardly true:
- Blue BIC pens exist in the world.
- There is a packet of blue BIC pens in Bob's house.
- Bob can prove the existence of blue BIC pens by showing them to Alice.
- Bob remembers the packet of blue BIC pens being in the drawer.
- It would be surprising to Bob if there were no packet of blue BIC pens in the drawer.
I'm okay with assigning "basically full credit" for Bob's statement. I wouldn't call it "sloppy"; I wouldn't say it was "too enthusiastic"; I wouldn't call for a "weaker version" or "more carefully worded argument". It's okay to make mistakes, try to fix them, and learn from them too.
The statement that I say is false and bad is:
"All I really meant was that I had blue pens at my house"
To me, Bob's words here prioritize status-defending over clarity. They remind me of this from TurnTrout's opening post:
An author committed to clarity might say something like: "I can see how my words led you to believe X. To be clear, what I mean is Y." This response takes responsibility.
A status-defending author might say something more like: "You are wrong to read it as X. It obviously means Y, and you are being uncharitable." This response deflects responsibility.
I'm not clear what your position is exactly (and it may not be mattmacdermott's position). I guess one of:
- Bob's statement is literally false, but should be understood as something else (what?) that is true.
- Bob's statement is true, because the meaning of Bob's earlier statements retroactively changed when Bob opened the drawer.
- Bob's statement is false, but given the low everyday-life stakes, it's ok.
I have some sympathy with these positions, but I respectfully disagree.
Thanks for the elaboration. My position is that Bob's first statement is literally false but that it implies a truth and the implication is what has been important for Alice throughout the interaction; that Bob's second statement does NOT claim that the first one should have been originally understood as something else that is true - only in the explicit context of additional evidence; that Bob does NOT claim that his earlier statement retroactively changed.
I would still say that Bob did communicate that he was "too enthusiastic originally" and that he would have wanted to make "more carefully worded argument" if the stakes were higher - the one that was implied by the original statement and that would not have been literally false but would have been literally true if worded more carefully. But since the implied argument that he meant to say is clear to all participants at the time of the second statement, Bob knows that Alice knows what Bob meant to say, thus I say it's OK for Bob to say that the thing in his mind had the correct shape all along - in all important aspects given the stakes of the discussion - he really did mean the thing that they together discovered as the truth even if the exact thing he said turned out literally false, but it's false in a way that doesn't matter now that they both know better.
I believe both of us would agree that if Bob said "All I really meant to say was that I had blue pens at my house", that would have been a better way to comment on his own mistake, without skipping the part that disambiguates between the 2 interpretations: whether he believes in equivalence between 2 literal statements that are incompatible; or whether he would have wished to think about his words more carefully and not try to bet on a statement that is too narrow (a la the "introverted librarian" vs "librarian" fallacy which I forgot the name of - edit: conjunction fallacy)...
But I believe our disagreement is that I took 1 extra charitable step and I don't believe Bob did any status-defending at the cost of clarity. I think Bob was clear in his communication as long as I don't take an adversarial stance towards the imperfectly-worded "All I really meant was that I had blue pens at my house". That the imperfect wording is excusable given the low stakes.
Though I would not endorse similar imperfect wording in a scientific paper errata.
2
1After reading your comments and @Jiro 's below, and discussing with LLMs on various settings, I think I was too strong in saying "large demerit" and "get a grip, Bob". While it's notable to me that Bob expressed himself with a status-defending post-hoc rationalization, that's just one piece of weak evidence to Bob's overall character.
TurnTrout's essay includes this disclaimer:
The context for this essay is serious, high-stakes communication: papers, technical blog posts, and tweet threads.
This thread was started with a thought experiment about low-stakes verbal communication. I incorrectly applied intuitions from the opening essay outside of the intended context.
1You need to distinguish between errors that are of no significance and errors that are significant. Although Bob's words were not literally true, the error is not relevant to the proposition for which the statement was used as evidence. (That's what a nitpick means: caring about an error that is not relevant to the associated proposition.)
Bob's statement 1: "I literally have a packet of blue BIC pens in my desk drawer" was not literally true, and that error was not relevant to the proposition that BIC make blue pens. I'm okay with assigning "basically full credit" for that statement.
Bob's statement 2: "All I really meant was that I had blue pens at my house" is not literally true. For what proposition is that statement being used as evidence? I don't see an explicit one in mattmacdermott's hypothetical. It's not relevant to the proposition that BIC make blue pens. This is the statement for which I assigned a "large demerit for being untrustworthy about the meaning of his own words, in a low stakes situation where there was no reason to lie".
I don't think that Bob's statement 2 is an error of no significance. If I'm Alice, and Bob is my friend, then he apparently just lied to my face. Hopefully it's a one-off slip of the tongue, and not part of a pattern.
Bob’s statement 2: “All I really meant was that I had blue pens at my house” is not literally true. For what proposition is that statement being used as evidence?
-
It's not being used as evidence for anything.
-
"All I really meant" is a colloquial way of saying "the part relevant to the proposition in question was..." As such, it was in fact truthful.
1
1I agree that the authors have a responsibility to communicate clearly. I also think readers have an obligation to read charitably, simply because the author can't anticipate all possible audiences and interpretations. The question really is how large is the relative effort each is expected to take. And here, I'd argue, the fundamental asymmetry between writers and readers informs an answer: A single person writes, but an article is read by many, many more people (a known mantra from coding for code quality). Whether people think about it in this terms or not, processes tend to take this into account. Large newspapers used to have almost paranoid quality control to avoid even small inconveniences on readers.
Question to the moderators: How often is a LW post with karma 10, 50, 100, 200 typically read?
- ^
See the sources for "Code is written once but read many times."
1Independently of the broader point, here's some comments on the particular example from the Scientist AI paper (source: I am an author):
- reward tampering arguments were and are a topic of disagreement between the authors, some of whom have views similar to yours, and some of whom have views well-expressed by the quoted passage
- I predict the lead author (Yoshua) would indeed own that passage and not say, "That was sloppy, what we really meant was..."
- so while it's fine as an example of readers perhaps inappropriately substituting an interpretation that seems more correct to them, I think it's not great as an example of authors motte-and-baileying or whatever (although admittedly having a bunch of different authors on a document can push against precision in areas where there's less consensus)
Thanks for writing this up! I highly agree and I think this is an important point to emphasis. Stating crucial context/clarification/assumptions is important, and unclear communication is likely counter-productive.
Part of the problem IMO is that both sides of that conversation seem to be mainly engaging in a debate over “yay Bengio” versus “boo Bengio”. That kind of debate (I call it “vibes-based meaningless argument”) is not completely decision-irrelevant, but IMO people in general are motivated to engage in it way out of proportion to its very slight decision-relevance†. Better to focus on what’s really decision-relevant here: mainly “what is the actual truth about reward tampering?” (and perhaps “shall I email the authors and bug them to rewrite that section?” or “shall I publicly complain?”, but probably not, that’s pretty rare).
†For example, it’s slightly useful to have a “general factor of Bengio being correct about stuff”, since that can inform how and whether one will engage with other Bengio content in the future. But it’s not that useful, because really Bengio is gonna be correct about some things and incorrect about other things, just like everyone is.
Yes, that happens a lot. The question is then maybe a differential one: What is the responsibility of the author in a political post vs. one that tries to improve the discourse?
I think he's using sloppy language.
Bengio et al. mix up "the policy/AI/agent is trained with RL and gets a high (maximal?) score on the training distribution" and "the policy/AI/agent is trained such that it wants to maximize reward (or some correlates) even outside of training".
Wait, does the friend elsewhere add "... and the author is right" or "and sloppiness isn't that bad"? My read of the quote you've provided is a critique and isn't excusing the sloppiness.