Authors Have a Responsibility to Communicate Clearly

TurnTrout

LESSWRONG
LW

125 Authors Have a Responsibility to Communicate Clearly

by TurnTrout

1st Jul 2025

Linkpost from turntrout.com

7 min read

125

Communication CulturesCommunityRationality

Frontpage

125

New Comment

29 comments, sorted by

top scoring

Click to highlight new comments since: Today at 6:08 AM

[-]Kaj_Sotala14d3416

It used to be that I would sometimes read something and interpret it to mean X (sometimes, even if the author expressed it sloppily). Then I would say "I think the author meant X" and get into arguments with people who thought the author meant something different. These arguments would be very frustrating, since no matter how certain I was of my interpretation, short of asking the author there was no way to determine who was right.

At some point I realized that there was no reason to make claims about the author's intent. Instead of saying "I think the author meant X", I could just say "this reads to me as saying X". Now I'm only reporting on how I'm personally interpreting their words, regardless of what they might have meant. That both avoids pointless arguments about what the author really meant, and is more epistemically sensible, since in most cases I don't know that my reading of the words is what the author really intended.

Of course, sometimes I might have reason to believe that I do know the author's intent. For example, if I've spent quite some time discussing X with the author directly, and have a good understanding of how they think about the topic. In those cases I might still make claims of their intent. But generally I've stopped making such claims, which has saved me from plenty of pointless arguments.

[-]Vladimir_Nesov14d168

The useful thing is the ideas you take away, it's rarely relevant if the author intended them or not.

[-]habryka14d3221

In policy contexts (not just political policy, but like, making group decisions of almost any kind), it tends to matter a lot what the author intended, since that's often a natural schelling point for resolution of ambiguities that later gets referred to.

See for example courts trying to interpret what previous courts intended with a judgement, or what a law was intended to do. Same for company decisions to move ahead with a project. In almost any context with stakes, the intention of the author continues to matter (though how much varies from context to context, though my sense is almost always a good amount).

[-]iainlim13d10

Yes but it's also not rare for legal scholars to see "legislative intention" as nothing but a convenient fiction that courts employ to retain some semblance of objectivity. In principle, they could drop such language and refer directly to their interpretation of the statutory text, right? The intellectual labor involved is unchanged: each judge considers their community's surrounding norms and common sense assumptions to understand what a law is "really" trying to do. Whether or not this interpretation is ascribed to authorial intention is basically superfluous.

[-]Benquo12d40

At least some scholars see a meaningful difference between "original intent" and "original public meaning."

[-]plex13d50

This feels very flavoured like A Principled Cartoon Guide to NVC's view on avoiding communication failures!

[-]Neel Nanda14d337

This seems somewhat backwards to me. Communication is kind of like a tree search of meaning. Everything you say is full of ambiguity and you can spend more words clarifying them by choosing specific nodes of the tree. To dig into this means you need to prioritise which bits are important, you can't make everything precise. I generally disapprove of critiquing people who don't put in the effort to clearly communicate a thing that was clearly unimportant to that point and I view the sloppiness defence as saying that "you are making an implicit argument that this thing is low quality because it has an error. However, the error is in a part of the tree that wasn't very relevant for the arguments. So who cares"

Separately, there is someone who supports their case with reasoning that is invalid and ambiguity defends them from people making this critique. But there, even if you give them the benefit of the doubt, the updated argument does not support the conclusion, so sloppiness is irrelevant - their argument is invalid

Maybe there's a gray zone where the detail seems plausibly relevant to a reader but actually isn't? Critique seems reasonable here.

I don't know which category the Bengio thing falls into - it seems like it makes an invalid argument to support a specific point in its overall thesis? I haven't read it

[-][anonymous]14d178

Regardless of who was "right", this conversation is a springboard for examining the broader conversational move of claiming "the author was right but was writing sloppily."

Without getting into the specifics of this particular example,^[1] sometimes this pattern occurs when an author wants to communicate some intuitive conclusion they have reached, but that intuition is the result of interacting such a large amount of literature/research/empirics/self-reflection/personal experience etc. that they can no longer point to any single short and compelling argument for why it's true.

Mindful of norms and razors such as "what can be asserted without evidence can also be dismissed without evidence," they believe saying "I believe X because of intuition, but I can't explain why" would be dismissed by their audience,^[2] so they instead try to manufacture some makeshift argument for it because it feels more epistemically virtuous^[3] to come up with "proper" scientific evidence. Unfortunately, their argument is often subtly wrong or slightly off-topic, which shouldn't be surprising; after all, it's not the argument that actually caused them to believe their conclusion, but more akin to a post-hoc confabulation.

See also Kaj Sotala's comment here and Ben Pace's summary of Jan Kulveit's comment here.

^{^}
Which I acknowledge at the outset doesn't actually fit into what I'm describing here
^{^}
Which isn't entirely unreasonable to do if you're such an audience member, but certainly not the optimal action to take if you have some degree of trust in the competence of the speaker's intuition
^{^}
Even though it isn't

[-]AnthonyC14d61

I think this is an important point, especially when experts are talking to other experts about their respective fields. I once had a client call this "thinking in webs." If you have a conclusion that you reached via a bunch of weak pieces of evidence collected over a bunch of projects and conversations and things you've read all spread out over years, it might or might not be epistemically correct to add those up to a strong opinion. But, there may be literally no verbally compelling way to express the source of that certainty. If you try, you'll have forgotten most of the evidence, and even if you don't, you'll end up with an audience that's bored to tears and (even if they're trying to understand) able to easily explain away each individual piece of evidence.

One canonical example might be Niels Bohr telling Einstein to quit telling God what to do. Not everything Einstein thought was right, but he clearly had and used very strong intuitions, based on thinking about known examples, regarding the nature of physical law. These were difficult to express and (rightly) would not, themselves, be considered scientifically valid evidence for drawing conclusions. IIRC I think that specific retort was in regards to "God does not play dice," which AFAIK most physicists then and since have disagreed with. But, depending on what specifically Einstein meant, and how the universe is actually operationalizing the underlying physics we see as quantum mechanics, it is still entirely possible he was right, but in a different way then he would have expected at the time.

[-]Jozdien15d136

Thanks for the post! I think there are particularly egregious examples where this clearly holds—e.g., I think it's probably good to have a norm that when pushed on something specific that's ambiguous authors say "[This] is what I meant", instead of constantly evading being pinned down.

That said, I think there are many examples of essays where authors just did not foresee a potential interpretation or objection (an easy example is an essay written years before such discussion). Such writing could conflate things because it didn't seem important not to—I think precision to the degree of always forestalling this is ~infeasible. I've talked to people who point at clear examples of this as examples of sloppy writing or authors reneging on this responsibility, when I think that particular ask is far too high.

To be clear, this is to forestall what I think is a potential interpretation of this post, not to imply that's what you meant.

[-]mattmacdermott14d123

Not sure how to think about this overall. I can come up with examples where it seems like you should assign basically full credit for sloppy or straightforwardly wrong statements.

E.g. suppose Alice claims that BIC only make black pens. Bob says, "I literally have a packet of blue BIC pens in my desk drawer. We will go to my house, open the drawer, and you will see them." They go to Bob's house, and lo, the desk drawer is empty. Turns out the pens are on the kitchen table instead. Clearly it's fine for Bob to say, "All I really meant was that I had blue pens at my house, the point stands."

I think your mention of motte-and-baileys probably points at the right refinement: maybe it's fine to be sloppy if the version you later correct yourself to has the same implications as what you literally said. But if you correct yourself to something easier to defend but that doesn't support your initial conclusion to the same extent, that's bad.

EDIT: another important feature of the pens example is that the statement Bob switched to is uncontroversially true. If on finding the desk drawer empty he instead wanted to switch to, "I left them at work", then probably he should pause and admit a mistake first.

[-]Martin Randall14d111

Based on your description of that incident, I expect that when Bob said:

"I literally have a packet of blue BIC pens in my desk drawer. We will go to my house, open the drawer, and you will see them."

that he did not, in fact, really mean:

"I have blue pens at my house"

That is a bizarre interpretation of that set of words. Why did Alice and Bob open the drawer to look for pens if they didn't both interpret his words as indicating that there was, literally, a packet of blue BIC pens in Bob's desk drawer?

So when Bob says:

"All I really meant was that I had blue pens at my house"

then he is making a false statement. Perhaps he is knowingly making a false statement - that's ethically bad. Perhaps he is unable to comprehend the meaning of his own words - that's epistemically bad. Either way it's not fine.

Instead Bob could say a true thing like:

"I was wrong about where the pens were, but right about BIC making blue pens"

Or a shorter true thing like:

"See, BIC makes blue pens".

My score for Bob:

full credit for knowing whether BIC makes blue pens.
partial credit for mostly knowing the location of his pens.
large demerit for being untrustworthy about the meaning of his own words, in a low stakes situation where there was no reason to lie. Get a grip, Bob.

[-]Aprillion12d96

huh? I see the bizarreness in the opposite way - why would you score Bob in such an anti-useful way?

to me, it seems blindingly obvious what Bob was trying to communicate and that Alice would be completely on Bob's side without any untrustworthiness points about any words or anything else between them, not even a hint... they looked into the drawer because that's where Bob remembered leaving them, then they both concluded that the exact statement is wrong and that they should focus on the gist of the problem they are trying to solve - if they both see the blue pens on the table, they both agree about the existence of blue BIC pens and they both agree that either Bob's memory is faulty or that someone else moved the pens, but that it doesn't matter, that the argument Bob should have made was a weaker version of what he actually said when he was too enthusiastic, that a more carefully worded argument would have been warranted but that what he meant was that he can prove the existence of blue pens by showing them to Alice

I believe Bob has a grip, and that Alice does not even need to be that extra charitable, just reasonable / not adversarial. There is no need to paint Bob in any wrong light here, the structure of this story is completely within the bounds of intellectual standards proposed in the OP, I don't see what could possibly be said against Bob here in any honest manner.

If Bob's prediction was written down "for the record" somewhere, that place should be updated by the new definition that both Bob and Alice completely agree about, for the benefit of onlookers - yes, I can agree with that! But in this story, Bob should NOT be accused of anything - not even that he was sloppy - given the everyday-life stakes of this story, the level of precision was entirely appropriate IMHO.

[-]Martin Randall12d139

We may have different values, or we may be imagining a different hypothetical, or something else. I'll try to elaborate.

Like you, it seems blindingly obvious to me what Bob was trying to communicate by saying "I literally have a packet of blue BIC pens in my desk drawer". To be explicit, Bob is stating his belief that there is a packet of blue BIC pens in his desk drawer. He doesn't specify his credence, but it's probably somewhere 90-99% depending on other factors.

Bob's words imply other things that have been mentioned, all of which are straightforwardly true:

Blue BIC pens exist in the world.
There is a packet of blue BIC pens in Bob's house.
Bob can prove the existence of blue BIC pens by showing them to Alice.
Bob remembers the packet of blue BIC pens being in the drawer.
It would be surprising to Bob if there were no packet of blue BIC pens in the drawer.

I'm okay with assigning "basically full credit" for Bob's statement. I wouldn't call it "sloppy"; I wouldn't say it was "too enthusiastic"; I wouldn't call for a "weaker version" or "more carefully worded argument". It's okay to make mistakes, try to fix them, and learn from them too.

The statement that I say is false and bad is:

"All I really meant was that I had blue pens at my house"

To me, Bob's words here prioritize status-defending over clarity. They remind me of this from TurnTrout's opening post:

An author committed to clarity might say something like: "I can see how my words led you to believe X. To be clear, what I mean is Y." This response takes responsibility.

A status-defending author might say something more like: "You are wrong to read it as X. It obviously means Y, and you are being uncharitable." This response deflects responsibility.

I'm not clear what your position is exactly (and it may not be mattmacdermott's position). I guess one of:

Bob's statement is literally false, but should be understood as something else (what?) that is true.
Bob's statement is true, because the meaning of Bob's earlier statements retroactively changed when Bob opened the drawer.
Bob's statement is false, but given the low everyday-life stakes, it's ok.

I have some sympathy with these positions, but I respectfully disagree.

[-]Aprillion12d50

Thanks for the elaboration. My position is that Bob's first statement is literally false but that it implies a truth and the implication is what has been important for Alice throughout the interaction; that Bob's second statement does NOT claim that the first one should have been originally understood as something else that is true - only in the explicit context of additional evidence; that Bob does NOT claim that his earlier statement retroactively changed.

I would still say that Bob did communicate that he was "too enthusiastic originally" and that he would have wanted to make "more carefully worded argument" if the stakes were higher - the one that was implied by the original statement and that would not have been literally false but would have been literally true if worded more carefully. But since the implied argument that he meant to say is clear to all participants at the time of the second statement, Bob knows that Alice knows what Bob meant to say, thus I say it's OK for Bob to say that the thing in his mind had the correct shape all along - in all important aspects given the stakes of the discussion - he really did mean the thing that they together discovered as the truth even if the exact thing he said turned out literally false, but it's false in a way that doesn't matter now that they both know better.

I believe both of us would agree that if Bob said "All I really meant to say was that I had blue pens at my house", that would have been a better way to comment on his own mistake, without skipping the part that disambiguates between the 2 interpretations: whether he believes in equivalence between 2 literal statements that are incompatible; or whether he would have wished to think about his words more carefully and not try to bet on a statement that is too narrow (a la the "introverted librarian" vs "librarian" fallacy which I forgot the name of - edit: conjunction fallacy)...

But I believe our disagreement is that I took 1 extra charitable step and I don't believe Bob did any status-defending at the cost of clarity. I think Bob was clear in his communication as long as I don't take an adversarial stance towards the imperfectly-worded "All I really meant was that I had blue pens at my house". That the imperfect wording is excusable given the low stakes.

Though I would not endorse similar imperfect wording in a scientific paper errata.

[-]Martin Randall2d21

After reading your comments and @Jiro 's below, and discussing with LLMs on various settings, I think I was too strong in saying "large demerit" and "get a grip, Bob". While it's notable to me that Bob expressed himself with a status-defending post-hoc rationalization, that's just one piece of weak evidence to Bob's overall character.

TurnTrout's essay includes this disclaimer:

The context for this essay is serious, high-stakes communication: papers, technical blog posts, and tweet threads.

This thread was started with a thought experiment about low-stakes verbal communication. I incorrectly applied intuitions from the opening essay outside of the intended context.

[-]Jiro9d31

You need to distinguish between errors that are of no significance and errors that are significant. Although Bob's words were not literally true, the error is not relevant to the proposition for which the statement was used as evidence. (That's what a nitpick means: caring about an error that is not relevant to the associated proposition.)

[-]Martin Randall5d55

Bob's statement 1: "I literally have a packet of blue BIC pens in my desk drawer" was not literally true, and that error was not relevant to the proposition that BIC make blue pens. I'm okay with assigning "basically full credit" for that statement.

Bob's statement 2: "All I really meant was that I had blue pens at my house" is not literally true. For what proposition is that statement being used as evidence? I don't see an explicit one in mattmacdermott's hypothetical. It's not relevant to the proposition that BIC make blue pens. This is the statement for which I assigned a "large demerit for being untrustworthy about the meaning of his own words, in a low stakes situation where there was no reason to lie".

I don't think that Bob's statement 2 is an error of no significance. If I'm Alice, and Bob is my friend, then he apparently just lied to my face. Hopefully it's a one-off slip of the tongue, and not part of a pattern.

[-]Jiro3d20

Bob’s statement 2: “All I really meant was that I had blue pens at my house” is not literally true. For what proposition is that statement being used as evidence?

It's not being used as evidence for anything.
"All I really meant" is a colloquial way of saying "the part relevant to the proposition in question was..." As such, it was in fact truthful.

[-]Gunnar_Zarncke15d104

I agree that the authors have a responsibility to communicate clearly. I also think readers have an obligation to read charitably, simply because the author can't anticipate all possible audiences and interpretations. The question really is how large is the relative effort each is expected to take. And here, I'd argue, the fundamental asymmetry between writers and readers informs an answer: A single person writes, but an article is read by many, many more people (a known mantra from coding for code quality). Whether people think about it in this terms or not, processes tend to take this into account. Large newspapers used to have almost paranoid quality control to avoid even small inconveniences on readers.

Question to the moderators: How often is a LW post with karma 10, 50, 100, 200 typically read?

^{^}
See the sources for "Code is written once but read many times."

[-]mattmacdermott14d92

Independently of the broader point, here's some comments on the particular example from the Scientist AI paper (source: I am an author):

reward tampering arguments were and are a topic of disagreement between the authors, some of whom have views similar to yours, and some of whom have views well-expressed by the quoted passage
I predict the lead author (Yoshua) would indeed own that passage and not say, "That was sloppy, what we really meant was..."
so while it's fine as an example of readers perhaps inappropriately substituting an interpretation that seems more correct to them, I think it's not great as an example of authors motte-and-baileying or whatever (although admittedly having a bunch of different authors on a document can push against precision in areas where there's less consensus)

[-]wonder14d60

Thanks for writing this up! I highly agree and I think this is an important point to emphasis. Stating crucial context/clarification/assumptions is important, and unclear communication is likely counter-productive.

[-]Steven Byrnes14d*40

Part of the problem IMO is that both sides of that conversation seem to be mainly engaging in a debate over “yay Bengio” versus “boo Bengio”. That kind of debate (I call it “vibes-based meaningless argument”) is not completely decision-irrelevant, but IMO people in general are motivated to engage in it way out of proportion to its very slight decision-relevance†. Better to focus on what’s really decision-relevant here: mainly “what is the actual truth about reward tampering?” (and perhaps “shall I email the authors and bug them to rewrite that section?” or “shall I publicly complain?”, but probably not, that’s pretty rare).

†For example, it’s slightly useful to have a “general factor of Bengio being correct about stuff”, since that can inform how and whether one will engage with other Bengio content in the future. But it’s not that useful, because really Bengio is gonna be correct about some things and incorrect about other things, just like everyone is.

[-]Gunnar_Zarncke14d20

Yes, that happens a lot. The question is then maybe a differential one: What is the responsibility of the author in a political post vs. one that tries to improve the discourse?

[-]DanielFilan14d33

I think he's using sloppy language.
Bengio et al. mix up "the policy/AI/agent is trained with RL and gets a high (maximal?) score on the training distribution" and "the policy/AI/agent is trained such that it wants to maximize reward (or some correlates) even outside of training".

Wait, does the friend elsewhere add "... and the author is right" or "and sloppiness isn't that bad"? My read of the quote you've provided is a critique and isn't excusing the sloppiness.

[-]Gavin Runeblade14d21

Eschew obfuscation. -- Mark Twain

[-]Said Achmiz14d*20

125 Authors Have a Responsibility to Communicate Clearly

by TurnTrout

1st Jul 2025

Linkpost from turntrout.com

7 min read

125

Communication CulturesCommunityRationality

Frontpage

125

New Comment

29 comments, sorted by

top scoring

Click to highlight new comments since: Today at 6:08 AM

[-]Kaj_Sotala14d3416

[-]Vladimir_Nesov14d168

The useful thing is the ideas you take away, it's rarely relevant if the author intended them or not.

[-]habryka14d3221

[-]iainlim13d10

[-]Benquo12d40

At least some scholars see a meaningful difference between "original intent" and "original public meaning."

[-]plex13d50

This feels very flavoured like A Principled Cartoon Guide to NVC's view on avoiding communication failures!

[-]Neel Nanda14d337

Maybe there's a gray zone where the detail seems plausibly relevant to a reader but actually isn't? Critique seems reasonable here.

I don't know which category the Bengio thing falls into - it seems like it makes an invalid argument to support a specific point in its overall thesis? I haven't read it

[-][anonymous]14d178

Regardless of who was "right", this conversation is a springboard for examining the broader conversational move of claiming "the author was right but was writing sloppily."

See also Kaj Sotala's comment here and Ben Pace's summary of Jan Kulveit's comment here.

^{^}
Which I acknowledge at the outset doesn't actually fit into what I'm describing here
^{^}
Which isn't entirely unreasonable to do if you're such an audience member, but certainly not the optimal action to take if you have some degree of trust in the competence of the speaker's intuition
^{^}
Even though it isn't

[-]AnthonyC14d61

[-]Jozdien15d136

To be clear, this is to forestall what I think is a potential interpretation of this post, not to imply that's what you meant.

[-]mattmacdermott14d123

Not sure how to think about this overall. I can come up with examples where it seems like you should assign basically full credit for sloppy or straightforwardly wrong statements.

[-]Martin Randall14d111

Based on your description of that incident, I expect that when Bob said:

"I literally have a packet of blue BIC pens in my desk drawer. We will go to my house, open the drawer, and you will see them."

that he did not, in fact, really mean:

"I have blue pens at my house"

So when Bob says:

"All I really meant was that I had blue pens at my house"

Instead Bob could say a true thing like:

"I was wrong about where the pens were, but right about BIC making blue pens"

Or a shorter true thing like:

"See, BIC makes blue pens".

My score for Bob:

full credit for knowing whether BIC makes blue pens.
partial credit for mostly knowing the location of his pens.
large demerit for being untrustworthy about the meaning of his own words, in a low stakes situation where there was no reason to lie. Get a grip, Bob.

[-]Aprillion12d96

huh? I see the bizarreness in the opposite way - why would you score Bob in such an anti-useful way?

[-]Martin Randall12d139

We may have different values, or we may be imagining a different hypothetical, or something else. I'll try to elaborate.

Bob's words imply other things that have been mentioned, all of which are straightforwardly true:

Blue BIC pens exist in the world.
There is a packet of blue BIC pens in Bob's house.
Bob can prove the existence of blue BIC pens by showing them to Alice.
Bob remembers the packet of blue BIC pens being in the drawer.
It would be surprising to Bob if there were no packet of blue BIC pens in the drawer.

The statement that I say is false and bad is:

"All I really meant was that I had blue pens at my house"

To me, Bob's words here prioritize status-defending over clarity. They remind me of this from TurnTrout's opening post:

An author committed to clarity might say something like: "I can see how my words led you to believe X. To be clear, what I mean is Y." This response takes responsibility.

A status-defending author might say something more like: "You are wrong to read it as X. It obviously means Y, and you are being uncharitable." This response deflects responsibility.

I'm not clear what your position is exactly (and it may not be mattmacdermott's position). I guess one of:

Bob's statement is literally false, but should be understood as something else (what?) that is true.
Bob's statement is true, because the meaning of Bob's earlier statements retroactively changed when Bob opened the drawer.
Bob's statement is false, but given the low everyday-life stakes, it's ok.

I have some sympathy with these positions, but I respectfully disagree.

[-]Aprillion12d50

Though I would not endorse similar imperfect wording in a scientific paper errata.

[-]Martin Randall2d21

TurnTrout's essay includes this disclaimer:

The context for this essay is serious, high-stakes communication: papers, technical blog posts, and tweet threads.

This thread was started with a thought experiment about low-stakes verbal communication. I incorrectly applied intuitions from the opening essay outside of the intended context.

[-]Jiro9d31

[-]Martin Randall5d55

[-]Jiro3d20

Bob’s statement 2: “All I really meant was that I had blue pens at my house” is not literally true. For what proposition is that statement being used as evidence?

It's not being used as evidence for anything.
"All I really meant" is a colloquial way of saying "the part relevant to the proposition in question was..." As such, it was in fact truthful.

[-]Gunnar_Zarncke15d104

Question to the moderators: How often is a LW post with karma 10, 50, 100, 200 typically read?

^{^}
See the sources for "Code is written once but read many times."

[-]mattmacdermott14d92

Independently of the broader point, here's some comments on the particular example from the Scientist AI paper (source: I am an author):

reward tampering arguments were and are a topic of disagreement between the authors, some of whom have views similar to yours, and some of whom have views well-expressed by the quoted passage
I predict the lead author (Yoshua) would indeed own that passage and not say, "That was sloppy, what we really meant was..."
so while it's fine as an example of readers perhaps inappropriately substituting an interpretation that seems more correct to them, I think it's not great as an example of authors motte-and-baileying or whatever (although admittedly having a bunch of different authors on a document can push against precision in areas where there's less consensus)

[-]wonder14d60

[-]Steven Byrnes14d*40

[-]Gunnar_Zarncke14d20

Yes, that happens a lot. The question is then maybe a differential one: What is the responsibility of the author in a political post vs. one that tries to improve the discourse?

[-]DanielFilan14d33

I think he's using sloppy language.
Bengio et al. mix up "the policy/AI/agent is trained with RL and gets a high (maximal?) score on the training distribution" and "the policy/AI/agent is trained such that it wants to maximize reward (or some correlates) even outside of training".

Wait, does the friend elsewhere add "... and the author is right" or "and sloppiness isn't that bad"? My read of the quote you've provided is a critique and isn't excusing the sloppiness.

[-]Gavin Runeblade14d21

Eschew obfuscation. -- Mark Twain

[-]Said Achmiz14d*20

A case study of the “sloppy language” move

This essay is not about AI alignment per se. I just use an example from AI alignment. My point stands regardless of your views on alignment.

I recently read Bengio et al.’s “Scientist AI” proposal. They argue that we are not currently building AI in a way which can reasonably be made safe. To support this argument, they claim that reinforcement learning will train agents which want to maximize their "reward."

Bengio et al. criticize RL to justify their proposal (§ 2.4.5)

Optimality of reward tampering. We now make the argument that reward tempering is not merely a fantastical scenario that we must guard against (although it certainly appears that way), but also a uniquely rational solution for an agent that takes reward maximization seriously. Before we begin, it is important to note that once an RL agent is trained, it continues trying to act so as to maximize the rewards it anticipates would come based on its training, even if the rewards actually never come in deployment. If it has a good model of the world and sufficient reasoning abilities, it can generalize from the circumstances in which it received large rewards in the past to new circumstances, by reasoning about the consequences of its actions.

I think the authors are wrong about that, as I have discussed at length elsewhere. I won't rehash those arguments because – for this essay – it doesn't matter if I'm right. Instead, consider that I forwarded this excerpt to a friend who claimed that Bengio et al. were just using "sloppy language." He argued that they didn't really mean that the RL agent will "act so as to maximize the rewards" – instead, they actually "meant that the RL agent will maximize reward during its training process."

What my friend said about the passage from Bengio et al.

I think he's using sloppy language.

Bengio et al. mix up "the policy/AI/agent is trained with RL and gets a high (maximal?) score on the training distribution" and "the policy/AI/agent is trained such that it wants to maximize reward (or some correlates) even outside of training".

Regardless of who was "right", this conversation is a springboard for examining the broader conversational move of claiming "the author was right but was writing sloppily."

Why the “sloppiness” move is harmful

While this move seems charitable, I argue that it’s actually harmful because: 1) it has the same negative impact as a wrong claim; 2) it erodes the meaning of our words; 3) it shields authors from their core responsibility; and 4) it serves as a social shield for dishonesty.

1. Unclear claims damage understanding

If some readers understand the "sloppy" language to advocate the wrong claim, then they have made the same belief update they would have if the author had meant to advocate the wrong claim. After all, the writer's intent stopped mattering the moment that editing ended and that publication began.

The reader’s mind does not know whether someone meant to miscommunicate an incorrect claim. All they read is the claim as written. Therefore, wrong claims and “sloppy” language have similar impacts on some readers.

Context clues help, but only if the target audience knows what to look for. If e.g. Yoshua's (secretly) correct argument needs to be guessed by knowing rare background knowledge about what the “sloppy language” is supposed to mean – then the context clues have failed.

Real people change their real minds based on these "unclear" claims. Many folks reallocate hundreds of hours of their professional lives to new problems. I was one of them. Since I was confused during my PhD, I spent thousands of hours on proving "power-seeking" theorems for reward-maximizing agents – theorems which I now consider interesting but misguided.

Making wrong claims imposes costs on others. To discourage the imposition of these costs, we tax or punish them. I'm not talking about anything crazy. If someone keeps saying weird stuff, and then they retreat to "it was just sloppiness", then that person takes a hit in my book.

2. Secret indirection erodes the meaning of language

If someone goes around saying X (wrong at face value) while meaning Y, then that sends the message “no, you should not read or speak plainly.” Words mean things.

3. Authors owe readers clarity

So don't misapply the principle of charity!

Imagine you ask someone for directions to the library. They provide a set of confusing instructions that lead you to the post office. When you inform them, they reply, "Oh, I knew the correct route. You just misinterpreted my directions."

While you have a responsibility to listen carefully, they have a responsibility to provide a clear map. Here, the "sloppiness move" allows the map-maker to blame the map-reader for a faulty map. This move rejects a core principle of the partnership: that the author has a responsibility to communicate effectively.

Yet I often see a hesitation to hold authors to this principle. I fear that my friend shirks from the perceived arrogance of declaring "yes, Bengio got this one wrong" or "even if they were being sloppy, that was bad." Their hesitation is understandable. Is it not kinder and more productive to interpret arguments in their strongest possible form?

The principle of charity requires interpreting a speaker's statements in the most rational way possible and, in the case of any argument, considering its best, strongest possible interpretation. In its narrowest sense, the goal of this methodological principle is to avoid attributing irrationality, logical fallacies, or falsehoods to the others' statements, when a coherent, rational interpretation of the statements is available.
— Wikipedia

The principle of charity is indeed valuable. However, the principle does not support the "sloppiness" move. The move doesn't ask us to interpret the author's words charitably. The move instead asks us to ignore their words and instead read their mind (relative to the expected knowledge of the target audience). We are asked to invent an entirely new interpretation of the author's statements – to invent an implausible interpretation not suggested by the text. Charity should be a tool for finding the truth, not for protecting an author's reputation.

But which interpretations are "plausible"?

Good-faith readers sometimes disagree about what interpretations are suggested by the text. This essay does not resolve that potential disagreement. However, whether or not they agree, I offer a clear prescription:

If people agree that an interpretation is not suggested by the text, then they should not entertain the idea that the author "actually" meant that interpretation.
If people disagree whether an interpretation is suggested by the text, then the author should clarify the intended interpretation.
- An author committed to clarity might say something like: "I can see how my words led you to believe X. To be clear, what I mean is Y." This response takes responsibility.
- A status-defending author might say something more like: "You are wrong to read it as X. It obviously means Y, and you are being uncharitable." This response deflects responsibility.

In neither case should the readers be forced to mind-read or provide complicated textual analyses justifying the "true meaning" of a passage.

4. The move can shield dishonesty

In the best case, the "sloppiness" statement is an attempt at charity. But this move also creates an opening for the worst case. When a dishonest actor is caught making a wrong claim, they can protest they were just “speaking sloppily.” This excuse is unfalsifiable and costless. By tolerating this move for the well-intentioned, we make it readily available for the dishonest.^[2]

Conclusion: Defending intellectual standards

Serious, high-stakes writing forms a bond of trust between the writer and the reader. To honor this bond, an author must take responsibility for the ideas their words convey. The “sloppiness” move, by shifting that responsibility onto the reader, threatens to break that trust.

By refusing this move, we do not refuse sloppy writing itself, nor do we encourage hostile, pedantic nitpicking of all possible ambiguities. We may instead adopt a simple code of conduct:

Writers: Own your words. When you miscommunicate, issue a clarification. A response that primarily blames the reader fails this standard.

Readers: Read in good faith, resolving minor ambiguities using context and extending reasonable charity to the author. Politely request clarification from the author but resist the urge to rationalize a "correct" interpretation that the text does not support. Furthermore, don't grant full credit for an idea that was “sloppily” communicated.

The proposed standard is simple: to speak clearly and to interpret reasonably. Expecting this of ourselves and of others is not an attack. It is a form of respect for our shared pursuit of truth.

Thanks to Guive Assadi, Peter Barnett, Alex Cloud, Rocket Drew, Gemini Pro 2.5, and Siao Si for giving feedback on drafts.

This article is readable in a prettier form on my website

Find out when I post more content: newsletter & RSS

Thoughts? Email me at alex@turntrout.com

^{^}
As noted by Guive Assadi: "It is efficient to allocate a lot of responsibility to the author because they are one person and if they work hard to write clearly, it saves many people time interpreting it."
^{^}
If used dishonestly, the "sloppiness" excuse functions as a motte-and-bailey. The (easy-to-defend) motte is the interpretation Y which the author "really" meant. The (hard-to-defend) bailey is X – one of the straightforward interpretations.