Yes, if you assume that the probability of seeing an observation was 100% under your favorite model then seeing it doesn't update you away from that model, but that assumption is obviously not true. (And I already conceded that the update is marginal!)

Alright -- apologies for the long delay, but this response meant I had to reread the Scaling Hypothesis post, and I had some motivation/willpower issues in the last week. But I reread it now.

I agree that the post is deliberately offensive at parts. E.g.:

But I think they lack a vision. As far as I can tell: they do not have any such thing, because Google Brain & DeepMind do not believe in the scaling hypothesis the way that Sutskever, Amodei and others at OA do. Just read through machine learning Twitter to see the disdain for the scaling hypothesis. (A quarter year on from GPT-3 and counting, can you name a single dense model as large as the 17b Turing-NLG—never mind larger than GPT-3?)

Google Brain is entirely too practical and short-term focused to dabble in such esoteric & expensive speculation, although Quoc V. Le’s group occasionally surprises you.

or (emphasis added)

OA, lacking anything like DM’s long-term funding from Google or its enormous headcount, is making a startup-like bet that they know an important truth which is a secret: “the scaling hypothesis is true!” So, simple DRL algorithms like PPO on top of large simple architectures like RNNs or Transformers can emerge, exploiting the blessings of scale, and meta-learn their way to powerful capabilities, enabling further funding for still more compute & scaling, in a virtuous cycle. [...]

and probably the most offensive is the ending (wont quote to not clutter the reply, but it's in Critiquing the Critics, especially from "What should we think about the experts?" onward). You're essentially accusing all the skeptics of falling victim to a bundle of biases/signaling incentives, rather than disagreeing with you for rational reasons. So you were right, this is deliberately offensive.

But I think the answer to the question -- well actually let's clarify what we're debating, that might avoid miscommunication. You said this in your initial reply:

I can definitely say on my own part that nothing of major value I have done as a writer online—whether it was popularizing Bitcoin or darknet markets or the embryo selection analysis or writing 'The Scaling Hypothesis'—would have been done if I had cared too much about "vibes" or how it made the reader feel. (Many of the things I have written definitely did make a lot of readers feel bad. And they should have. There is something wrong with you if you can read, say, 'Scaling Hypothesis' and not feel bad. I myself regularly feel bad about it! But that's not a bad thing.) Even my Wikipedia editing earned me doxes and death threats.

So in a nutshell, I think we're debating something like "will what I advocate mean you'll be less effective as a writer" or more narrowly "will what I'm advocating for mean you couldn't have written really valuable past pieces like the Scaling Hypothesis". To me it still seems like the answer to both is a clear no.

The main thing is, you're treating my position as if it's just "always be nice", which isn't correct. I'm very utilitarian (about commenting and in general) (one of my main insights from the conversation with Zack is that this is a genuine difference). I've argued repeatedly that Said's comment is ineffective, basically because of what Scott said in How Not to Lose an Argument. It was obviously ineffective at persuading Gordon. Now Said argued that persuading the author isn't the point, which I can sort of grant, but I think it will be similarly ineffective for anyone sympathetic to religion for the same reasons. So it's not that I terminally value being nice,^[1] it's that being nice is generally instrumentally useful, and would have been useful in Said's case. But that doesn't mean it's necessarily always useful.

I want to call attention my rephrasing of Said's post. I still claim that this post would have been much more effective in criticizing Gordon's post. Gordon would have reacted in more constructive way, and again, I think everyone else who sympathizes with religion is essentially in the same position. This seems to me like a really important point.

So to clarify, I would not have objected to the Scaling Hypothesis post despite some rudeness. The rudeness has a purpose (the bolded sentence is the one that I remembered most from reading it all the way back, which is evidence for your claim that "those were some of the most effective parts"). And the context is also importantly different; you're not directly replying to a skeptic; the post was likely to be read by lots of people who are undecided. And the fact that it was a super high effort post also matters because 'how much effort does the other person put into this conversation' is always one of the important parameters for vibes.

I also wanna point out that your response was contradictory in an important way. (This isn't meant as a gotcha, I think it capture the difference between "always be nice" and "maximize vibes for impact under the constraint of being honest and not misleading".) Because you said that you wouldn't have been successful if you worried about vibes, but also that you made the Scaling Hypothesis post deliberately offensive, which means you did care about vibes, you just didn't optimize them to be nice in this case.

Idk if this is worth adding, but two days ago I remembered something you wrote that I had mentally tagged as "very rude", and where following my principles would mean you're "not allowed" to write that. (So if you think that was important to write in this way, then we have a genuine disagreement.) That was your response to now-anonymous on your Clippy post, here. Here, my take (though I didn't reread, this is mostly from memory) is something like

• the critique didn't make a lot of sense because it boiled down to "you're asserting that people would do xyz, but xyz is stupid", which is a nonseqitor ("people do xyz" and "xyz is stupid" can both be true)
• your response was needlessly aggressive and you "lost" the argument in the sense that you failed the persuade the person who complained
• it was absolutely possible to write a better reply here; you could have just made the above point (i.e., "it being stupid doesn't mean it's unrealistic") in a friendly tone and the result would probably been that the commenter realizes their mistake; the same is achieved with fewer words and it arguably makes you look better. I don't see the downside.

The notion of evidence as rationalists use it is very expansive. B is evidnece for A if and only if $P\left(A|B\right)>P\left(A\right)$. Or equivalently, if and only if $P\left(A|\neg B\right)<P\left(A\right)$. If people never got bribed ($\neg B$), that sure seems to me like it would lower the probability of a deep state ($A$). Hence $P\left(A|\neg B\right)<P\left(A\right)$ and therefore $P\left(A|B\right)>P\left(A\right)$, which means people getting bribed is, in fact, evidence for a deep state.

(This is the same with people saying "correlation isn't evidence of causation". Incorrect; correlation is always evidence for causation, because lack of correlation is evidence against causation.)

Again, it's the magnitude of the evidence that you can dispute. If you think people will get bribed regardless, then people getting bribed is only very weak evidence for a deep state. But it's still evidence. Just like seeing bank robbers is evidence for a large society of bank robbers. (Because seeing no bank robbers would be evidence against such a society.)

I think it definitely is evidence for a deep state (if people couldn't get pressured or bribed, that looks to me like evidence against a deep state), the question is just how much.

I agree that unintuitiveness is a valid reason to reject the theory and the report; that doesn't contradict my comment.

This post looks to me like an extreme case of an isolated demand for rigor. People release reports with assumptions in them all the time. People release reports with not 100% rigorously defined terms in them all the time. Ditto for academic papers.

The fact that the authors of the report don't give us any proximate theories of consciousness, unfortunately, damns the whole project to h∄ll, which is where poor technical philosophies go when they make contact with reality (good technical philosophies stick around if they're true, or go to h∃aven if they're false).[3]

You have to provide a full theory of consciousness to release a report about priorities? Really? How is this different from arguing that Ajeya Cotra's report on AI timelines is worthless because it doesn't provide a theory of intelligence? (Not a definition of intelligence, mind you, but a theory?)

Unitarianism smuggles in an assumption of "amount" of valence, but the authors don't define what "amount" means in any way, not even to give competing theories of how to do so.

So the failure to define the word "amount" is the smoking gun? What if the report had added a fourth assumption

Quantifiability: valence can be quantified as a scalar number that behaves additively

Now they have defined what amount means. But they essentially did this anyway by saying that

There is an objective thing called 'valence' which we can assign to four-volumes of spacetime using a mathematical function (but we're not going to even speculate about the function here)

This essentially already says that they think valence, under the right theory of consciousness, amounts to a number. The implication is that once you have the right ToC, it will tell you how to quantify valence, and then you just add it up using addition.

Obviously you can disagree with these assumptions, and of course many people do. But the post accuses the report of "smuggling in an assumption about amount of valence". How is this smuggling in anything??? It's explicitly listed as an assumption.

I fail to see how this post provides any value beyond just stating that the author disagreed with the conclusions/assumptions of the report, and I think it's highly likely that the only reason it got upvoted is that most other people also disagreed with both the assumptions and the conclusions. I don't see how anyone who does agree with the assumptions could change their mind based on reading this, or how anyone who doesn't like the report couldn't have already explained why before reading this. Please tell me what I'm missing.

Portable AC is not annoying in any way I can perceive? My climates have not been very hot, but I consider AC to be a major factor in my quality of life

It's annoying for me because

• it's very loud (I bought a cheap model last time, probably expensive ones are less loud)
• sealing up the window is a pain
• once you seal up the window, you are limited in how much you can open it

I was still considering doing it; if I had foreseen how long the heatwave lasts, I'd have probably done it. But I don't consider it to be a huge deal.

You didn't say if you think this post is wrong or merely that this was already obvious to you. Or perhaps you think it's half right and obvious?

I think the general phenomenon of people suffering unnecessarily because they feel like it makes them virtuous is almost certainly real. I don't know how much of it applies to air conditioners. It wouldn't surprise me if it were a big part of the story, but it also wouldn't surprise me if it wasn't. I don't think it's that rare to have a general phenomenon and expect it to apply in a specific case, but then when you look at the data, that's not really born out, for whatever reason. I feel like I've seen this pattern a few times, where someone takes popular rationalist idea and asserts that it applies to situation X, and then someone else looks at the actual evidence and it turns out it doesn't really check out. (Arguably that was the case with the other air conditioner debate, although people still debated how much the data bore out the original take after the fact iirc. (To be clear, john used air conditioners to make a different point about consumer stupidity, not the principle of self-harm that's articulated here.))

Reporting personal impressions of attitudes is evidence of a sort. Survey responses are just about as difficult to interpret IMO.

I agree, but here I come back to, this is just not new at all. I'm pretty sure I've seen this general principle articulated somewhere in the sequences. Like if I had never heard of this principle, then it would be different. but if you're not saying anything new, and you're not being quantitative, what's the point? Especially on a topic where there is lots of relevant data you could dig up, it would just be a lot more effort.

Strong-downvoted this post because it doesn't provide any evidence for what it's arguing for, and also because it seems like a pretty safe take on LessWrong (which ups my standards for what I consider worthwhile a bit -- like I'd be more sympathetic to a zero-evidence post if the take were more out there). As is, it's firmly in the "I want to see less of this" category.

Also I live in Europe and don't have an AC because installing one properly is expensive and portable ones are annoying, and it altogether doesn't seem worth it.

No. But that's OK with me, because I don't regard "other people who use one of the same websites as me" as a generic authority figure.

Was definitely not going to make an argument from authority, just trying to understand your world view.

Iirc we've touched on four (increasingly strong) standards for truth

1. Don't lie
2. (I won't be the best at phrasing this) something like "don't try to make someone believe things for reasons that have nothing to do with why you believe it"
3. Use only the arguments that convinced you (the one you mentioned here
4. Make sure the comment accurately reflects your emotional state^[1] about the situation.

For me, I endorse #1, and about 80% endorse #2 (you said in an earlier comment that #1 is too weak, and I agree). #3 seems pretty bad to me because the most convincing arguments to me don't have to be the most convincing arguments the others (and indeed, they're often not), and the argument that persuaded me initially especially doesn't need to be good. And #4 seems extremely counter-productive both because it'll routinely make people angry and because so much of one's state of mind at any point is determined by irrelevant variables. It seems only slightly less crazy than -- and in fact very similar to -- the radical honesty stuff. (Only in the most radical interpretation of #4 is like that, but as I said in the footnote, the most radical interpretation is what you used when you applied it to Said's commenting style, so that's the one I'm using here.)

Here's an example from Ben Hoffman's "The Humility Argument for Honesty" [...]

This is not a useful example though because it doesn't differentiate between any two points on this 1-4 scale. You don't even need to agree with #1 to realize that trying to convince the doctor is a bad idea; all you need to do is realize that they're more competent than you at understanding symptoms. A non-naive purely impact based approach just describes symptoms honestly in this situation.

My sense is that examples that prefer something stronger than #2 will be hard to come up with. (Notably your argument for why a higher standard is better was itself consequentialist.)

Idk, I mean we've drifted pretty far off the original topic and we don't have to talk any more about this if you're not interested (and also you've already been patient in describing your model). I'm just getting this feeling -- vibe! -- of "hmm no this doesn't seem quite right, I don't think Zack genuinely believed #1-#4 all this time and everything was upstream of that, this position is too extreme and doesn't really align with the earliest comment about the moderation debate, I think there's still some misunderstanding here somewhere", so my instinct is to dig a little deeper to really get your position. Although I could be wrong, too. In any case, like I said, feel free to end the conversation here.

Minimalism certainly makes me happier. Not really because of any material cost/benefit analysis though, but just because it feels nicer to have a purpose for owning stuff. I like it that I can point at most things I own and be like "yeah I have that because xyz"