I feel like not publishing our private conversation (whether you're a journalist or not) falls under common courtesy or normal behaviour rather than "charity".

I feel like this falls into the fallacy of overgeneralization. "Normal" according to whom? Not journalists, apparently.

common courtesy is not the same as charity, and expecting it is not unreasonable.

It's (almost by definition) not unreasonable to expect common courtesy, it's just that people's definitions of what common courtesy even is vary widely. Journalists evidently don't think they're denying you common courtesy when they behave the way most journalists behave.

Standing more than a 1 centimeter away from you when talking is not charity just because it's technically legal - it's a normal and polite thing to do, so when someone comes super close to my face when talking I have the right to be surprised and protest. Escalating publicity is like escalating intimacy in this example.

This is an interesting pushback, but I feel the same reply works here: failing to respect someone's personal space is not inherently wrong, but it will be circumstantially wrong most of the time because it tends to do much more harm (i.e. annoy people) than good.

I don't think it's "charity" to increase the level of publicity of a conversation, whether digital or in person.

Neither do I: as I said, I actually think it's charity NOT to increase the level of publicity. And people are indeed charitable most of the time. I just think that, if you live your life expecting charity at every instance, you're in for a lot of disappointment, because even though most people are charitable most of the time, there's still going to be a lot of instances in which they won't be charitable. The OP seems to be taking charity for granted, and then complaining about a couple of instances in which it didn't happen. I think it's better to do the opposite: not to expect charity, and then be grateful when it does happen.

I think drawing a parallel with in person conversation is especially enlightening - imagine we were having a conversation in a room with CCTV (you're aware it's recorded, but believe it to be private). Me taking that recording and playing it on local news is not just "uncharitable" - it's wrong in a way which degrades trust.

I don't think it's inherently wrong. It may still be (and in most cases will be) circumstantially wrong, in the sense that it does much more damage to others (including, as you mention, by collaborating to degrade public trust) than it does good to anyone (yourself included).

I also don't think privacy is a binary.

That's an interesting perspective. I could subscribe to the idea that journalists may be missing the optimal point there, but that feels a bit weaker than your initial assertion.

Do you think that a conversation we have in LessWrong dms is as public as if I tweeted it?

I mean, I would not quote a DM without asking first. But I understand that as a kind of charity, not an ethical obligation, and while I try my best to be charitable towards others, I do not expect (nor do I feel in any way entitled to) the same level of compassion.

There's definitely a fair expectation against gossiping and bad-mouthing. I don't think that's quite what the OP is talking about, though. I believe the relevant distinction is that (generally speaking) those behaviors don't do any good to anyone, including the person spreading the gossip. But consider how murkier the situation becomes if you're competing for a promotion with the person here:

if you overheard someone saying something negative about their job and then going out of your way to tell their boss.

My understanding is that the OP is suggesting the journalists' attitude is unreasonable (maybe even unethical). You're saying that their attitude is justifiable because it benefits their readers. I don't quite agree that that reason is necessary, nor that it would be by itself sufficient. My view is that journalists are justified in quoting a source because anyone is generally justified in quoting what anyone else has actually said, including for reasons that may benefit no one but the quoter. There are certainly exceptions to this (if divulging the information puts someone in danger, for instance), but those really are exceptions, not the rule. The rule, as recognized both by common practice and by law, is that you simply have no general right to (or even expectation of) privacy about things you say to strangers, unless of course the parties involved agree otherwise.

This sounds absurd to me. Unless of course you're taking the "two golden bricks" literally, in which case I invite you to substitute it by "saving 1 billion other lives" and seeing if your position still stands.

I didn't downvote, but I would've hard disagreed on the "privacy" part if only there were a button for that. It's of course a different story if they're misquoting you, or taking quotes deliberately out of context to mislead. But to quote something you actually said but on second thought would prefer to keep out of publication is... really kind of what journalists need to do to keep people minimally well-informed. Your counterexamples involve communications with family and friends, and it's not very clear to me why the same heuristic should be automatically applied to conversations with strangers. But in any case, not even with the former your communication is "truly" private, as outside of very narrow exceptions like marital privilege, their testimony (on the record, for potentially thousands of people to read too) may be generally compelled under threat of arrest.

The problem here is that the set of all possible commands for which I can't (by that definition) be maximally rewarded is so vast that the statement "if someone maximally rewards/punishes you, their orders are your purpose of life" becomes meaningless.

Not true, as the reward could include all of the unwanted consequences of following the command being divinely reverted a fraction of a second later.

That’s a great question. If it turns out to be something like an LLM, I’d say probably yes. More generally, it seems to me at least plausible that a system capable enough to take over would also (necessarily or by default) be capable of abstract reasoning like this, but I recognize the opposite view is also plausible, so the honest answer is that I don’t know. But even if it is the latter, it seems that whether or not the system would have such abstract-reasoning capability is something at least partially within our control, as it’s likely highly dependent on the underlying technology and training.

To be rewarded (and even more so "maximally rewarded") is to be given something you actually want (and the reverse for being punished). That's the definition of what a reward/punishment is. You don't "choose" to want/not want it, any more than you "choose" your utility function. It just is what it is. Being "rewarded" with something you don't want is a contradiction in terms: at best someone tried to reward you, but that attempt failed.