The point is that "maintaining sanity" is a (much) higher bar than "Don't flail around like a drama queen". Maintaining sanity requires you to actually update on the situation you find yourself in, and continue to behave in ways that make sense given the reality as it looks after having updated on all the information available. Not matching obvious tropes of people losing their mind is a start, but it is no safe defense. Especially since not all repeated/noticeable failure modes are active and dramatic, and not all show up in fiction.
For example, if there's something to David Gross's comment that the wretched journalist was actually giving you an opening because they saw importance in what you had to say about the situation, blowing off a genuine opening to influence the discourse on AI safety while calling it "doing nothing" would not be sane. Preemptive contempt has a purpose in bounded rationality, but it's still a form of pushing away from the information the journalist has to offer. It can make sense within a grand plan that weights this journalist low, but that requires a grand plan.
How do you actually orient to the world, now that we are what we are? Are you still working to bring about the good outcome? If so, what's the grand plan that ties everything together? Sharing that seems important for helping people retain sanity. Have you given up? If so, what is the overarching plan that drives how you choose to interact with the world? Because you still have to decide what to do with your time.
This is a hell of a problem to orient to, and I don't know that any of us get to say we're doing it sanely. It's a high bar to strive towards.
The trope that this post and comment match to me isn't one that shows up in science fiction. It's a real bitch to wrestle free from, because the whole premise has to do with protecting stability of sense making by pushing away from challenging updates with avoidance and contempt, and the whole project fails if it doesn't turn meta and resist awareness of the trope. I notice that even writing and rewriting this comment to be minimally threatening of stability without holding back content, it's going to be a tough one to engage with to the extent that there isn't a preexisting superstructure regulating contact with reality to maintain stability while minimizing the cost of missed updates.
Which is certainly a possibility. As is leveraging the skill of becoming genre savvy as new patterns emerge ("trope dodging").
So if this contempt provokes contempt quickly, I'm sorry. My best isn't always good enough, which is kinda the possibility we're all wrestling with here.
Any thoughts on what to do if "just explain it to someone" turns into a long back and forth dialog?
The distinction between what might be called "lying" and "bullshitting" is important here, because they scale with competence differently.
It was pretty interesting watching this develop in my kids. Saying "No!" to "Did you take the cookie from the cookie jar?" is the first thing you get, because it doesn't require a concept for "truth" at all. Which utterance postpones trouble? Those are the sounds I shall make!
Yet for a while my wife and I were in a situation where we could just ask our younger kid about her fight with our older kid, because the younger kid did not have a concept for fabricating a story in order to mislead. She was developed enough to say "No!" to things she knew she did, but not developed enough to form the intention of misleading.
The impression I get from LLM is that they're bullshitters. Predict what text comes next. Reward the one's that sound good to some dumb human, and what's gonna come out? We don't need to postulate an intent to mislead, we just need to notice that there is no robust intent to maintain honest calibration -- which is hard. Much harder than "output text that sounds like knowing the answer".
It takes "growing up" to not default to bullshit out of incompetence. Whether we teach them they'll be rewarded by developing skillful and honest calibration, or by intentionally blowing smoke up our ass is another question.
I like this post because it takes things you can only learn by "actually doing things", and then presents them in ways that can be generalized.
My above description is false, actually. I've been saying that you are trying to hit the limit without going over. Actually, fast drivers hover at the limit. They oscillate between a little bit under and a little bit over. [...]
They find the limit by probing for it, dancing at it.
This part in particular, because the default assumption is "Oh no, can't cross the limit!", yet this is true about a lot of things.
Also, even if you're just driving to visit your grandma and not pushing the limits of traction, a traction aware driver will drive differently than your average driver. For example, it's quite common to approach a red light at their current driving speed, only to start braking harder and harder at the end. Which is a foolish use of the safety margin, and also slower than the person who brakes gently and early, and therefore is more likely to still have momentum when the light turns green.
The problem with talking about things is that we don't really have a good shared ontology of how "preferences"/"desires"/"values"/etc work, and they don't work the way people think they do.
Basically everything is way more context dependent than anyone realizes -- as in, "I only wanted to go to the store because I thought it had the food I wanted", to give a trivial example. But that food you had a preference for is subject to change as your bodies needs for nutrients changes. Even things like people's identities as "asexual" or "straight" are prone to update with the evidence we come across.
So then you try to say "Well, that's 'tastes', when I talk about 'values' I mean things like 'autonomy'". Except that kind of thing is merely instrumental as well -- stabilized by motivated blind spots about how useless autonomy can be in the right contexts. And then the right contexts come along, and your "values" shift. Which can sound like "Oh no! Value drift!" from the outside, but once you get there, it's just "Oh no, that store was closed. It's my recognition of that which has changed".
Then you try to retreat to "Okay, but pain is bad. Like, by definition!". Except it isn't, because masochists. Which aren't even uncommon, with how many people like spicy food, and hard massages that "hurt in a good way".
The last step seems to be avoidance of suffering, saying "Ah, right, pain isn't suffering" but suffering is the definition of bad!. Except we choose that too! Suffering is what we choose in order to stave off the loss of hope. Often without realizing it, so we can get stuck with unproductive suffering which really is good to eliminate, but it's something we choose nonetheless. And becoming conscious of it can allow more deliberate choice between hopelessness and continued suffering.
The whole thing is hard to make sense of, so it's kinda "Of course people are going to use terms in unclear and conflicting ways". When you say people should talk about things like "Their own preferences", are you referring to their preference to go to the store, or to eat the food that they believe the store has for them? Or something upstream of that? When you talk about "normative values", what the heck is that, exactly? If it's "The thing that we should value", then what exactly is that 'should' being used to distract from? Do we have any shared and accurate idea of what this means, descriptively speaking?
I think we need more deliberate study of how human tastes/desires/wants/values/etc change or don't, before we're going to have smooth hiccup-free communication on the topic. I agree with you that these terms conflate things, but I don't think we have the option of not conflating things yet. So I'm nudging away from "Just use clear language and then everything will be clear" and towards "notice what your concepts might be hiding, and how much ambiguity is necessarily left".
1While I understand the frustration, I'd rather have more hobby horse riders here. If I ever say something to inspire the charge of a hobby horse, I want that correction.
Because I might get lazy. Or imprecise. The "correction" might be something I immediately recognize as "obviously true", and want to say "Yeah yeah, that's what I meant". But it might not be what I said, and I may have been underweighting the importance of that little "nitpick" when I was writing. After all, that why there's the charging of the hobby horse; the other person doesn't think it's some unimportant nitpick. And neither do the LW voters, in the cases you highlight.
Maybe it's not.
If we try to discourage people from correcting real errors or misleading representations in the text, simply because the person pointing it out is unusually perceptive in this area, or is unusually aware of the importance of this kind of mistake, then we are in effect saying that we don't want to hear from people who are uniquely suited to correcting specific errors. "Sorry, Eliezer, you've been riding this AI hobby horse too much. We agree that making an unfriendly superintelligence would be bad, which is why we're going to make it friendly. Can't we move on and build it now?".
That doesn't cut it when the issue actually is important, and often the awareness of these things falls on few people. "What is a woman?" exploded into such a huge issue that I'm glad we have our resident "hobby horse rider" here, with skin in the game, motivated to do very careful thinking and call out what he sees to be errors on our part. If he's wrong he's wrong, which is a different criticism. If he's right though, I'd rather amend or clarify my writing to the satisfaction of the person who makes getting this particular thing right their thing. It might save me from mistakes I don't properly appreciate.
The qualifier "to the satisfaction of the other person" is important here. I know you think you've gotten things close enough. Likely so do the other authors in your examples. I also know that the hobby horse riding commenters disagree, and so does the audience -- at least in these cases. And that if you can't pass their ITT you can't know if you're missing something that validates their perspective and invalidates yours. And that if you can, they won't continue to think you don't get it, and therefore won't have reason to post those "unnecessary" comments.
I wonder how often questions like "What makes one race car driver faster than another" have a different answer from "What makes all race car drivers way faster than you".
I know from experience that "riding the limits of traction" is the first 90% that most people don't get, but how often is the last ten percent just chasing diminishing returns on the same thing, and how often is it a completely new skill that becomes relevant once you handle the "easy part"?
For example, using long range rifle shooting as an example, the answer to the former is "reading wind". But if you simply hand a rifle to someone who has never shot before, wind won't be the reason they miss. They still have to learn how to stabilize a rifle, calculate drop, etc.
But yes, interesting set of questions either way.
This is a normal consequence of intending at a level that requires more control than we actually have. Which is a normal consequence of not yet perceiving the interrelation and structure of expectation and control
When we control things, the effect of our control is to make our desired outcome expected -- for if we can't hit the center of the target even in expectation, then by definition we aren't in control. "Expecting" an outcome goes hand in hand with aiming to "control to" or "manifest" an expectation.
When the room is too cold, we think "Brr... it shouldn't be this cold in here!" and then go turn the heat up until room's temperature meets our expectations. Okay, fine.
But then what happens when your mom might have cancer?
You've been expecting her to not have cancer, and you want to be able to keep this expectation because who wants their mom to have cancer? So you might focus on the desired world state where your mom has no cancer, acting to do what you can to bring it about. You focus on manifesting no cancer in the biopsy -- and know this will fail, so you get this error signal that tells you it's not working in expectation. And then often in reality.
This resistance to letting go comes because we have something to lose. And there's something to fighting this fight. "Everything I've ever let go of has claw marks on it."
At the same time, it doesn't always work. And the suffering it entails points to our expectations actually being wrong. We're strongly expecting to not see cancer in the biopsy AND we know that this expectation is likely to be falsified. That hint we can update on.
I wish I could have certainty that my mom doesn't have cancer. Of course I wish that. Who wouldn't? At the same time, my mom might actually have cancer, and there ain't shit I can do about what's already true.
What I can do, is make sure her life does not get cut short unnecessarily. Not "My mom doesn't have cancer [dammit!]", but "My mom is going to live as long, healthy, and happy as a life is as absolutely possible. Because I'm going to make sure of it". I'm sure you, too, want to make sure your mom lives as long, healthy, and happily as absolutely possible. And you can act so as to make sure she does.
When that's your frame, where's the spider?
How do you feel about checking the biopsy, now?
For that matter, how do you feel about not checking the biopsy now?
Interesting, right?
So what do you do about the growing aversion to information which is unpleasant to learn?
To answer this directly, I notice. Like, really notice, and sit with it, and then notice what changes as a result as I realize what the implications are and allow the updates to flow through me.
Not "notice-and-then-do-this-instead!" because that's often prematurely jumping to try to a control a thing with insufficient perspective, when the problem itself is caused by trying to jump too quickly to control a thing without sufficient perspective.
So step one is to notice.
And to actively monitor whether I'm trying to "do something about it!", because I already know I don't want to jump to that. Not that I want to "Do-something-about-trying-to-do-something!", just "I don't want to do things that are stupid, lol".
Notice what the existence of this ugh field is telling me. Okay, I already know my expectations are bad. They won't be fulfilled, in my already existing meta-expectation.
What changes?
What doesn't?
Specifically, I look to what I'm realizing I can't control, and to what of value I still can control. And then reorient to that, so that I stop putting ineffectual claw marks on the things that's a goner at the expense of attending to what can still be saved.
So, "Hm. I notice that I don't want to see what's in this email, because I already suspect it will be what I don't want to see. Okay, what don't I want to see. Okay, yeah, I don't want to see that. Of course I don't want to see that. What if I do see that? What might I want to do about that"?
Maybe, "Why does it seem like whatever I do, people will get pissed at me?". "Is that actually true?". "If not, what kind of unseen-stupid am I being to systematically fail like this?". "If so, is that okay?".
The exact sequence and form might change, but the underlying theme is to be really attentive to what feedback I'm getting and where I might be flinching away from updating on this feedback, because all of this struggle results from failing to attend to something with the attention it deserves. The model I'm comparing to, to highlight sources of error, is one where my expectations aren't predictably violated, there's no innate tension underlying everything as a result, and any tension gets released by retreating from obstinate control towards more nuanced and obtainable goals after grieving what must be grieved -- and not what must not.
I see the point you're getting at, and I agree that there's a real failure mode here about I've been annoyed in similar ways. Heck, I kinda think it's silly for people to show up to promotions to receive the black belt they earned, but that's a separate topic.
At the same time, there's another side of this which is important.
At my jiu jitsu gym there's a new instructor who likes doing constraint led games. One of these games had the explicit goal of "get your opponents hands to the mat" with the implicit purpose of learning to off balance the top player. I decided to be a little muchkin and start grabbing peoples hands and pulling them to the mat even when they had a good base.
I actually did get social acclaim for this. The instructor thought that was awesome, and used it as an example of how he wanted people to play the games. In his view, as in mine, the point of the game is to explore how you can maneuver to win at the game as specified, without being restrained by artificial limitations which really ought to be accounted for in the game design.
If the new instructor would have tried to lecture us about playing to some underspecified "spirit" of the rules instead of the rules as he described them -- and about how we're not earning social points with him for gaming the system -- and was visibly annoyed about this... he would have been missing the point that he's not earning social points with me, and likely not with the others either. And I wouldn't much care for winning points with him, if that's how he were to respond. It's a filter. A feature, not a bug.
Breaking the game is to be encouraged, and if playing the game earnestly doesn't suit the intended purpose, "don't hate the player, hate the game". In his case, the game wasn't so broken so as to ruin the game so it turned out to be more fun and probably more useful than I had anticipated. Maybe it wasn't quite optimal, but it was playable for sure. In your case, the broken game is the sign that calibration isn't what we care about -- because that annoying shit was calibrated, and you weren't happy about it. What we need is a better scoring rule that weights calibration appropriately. Which exist!
Any time we find ourselves annoyed, there is a learning opportunity. Annoyance is our cue that reality is violating our expectations. It's a call to update.
In my experience, this isn't true because fighting back effectively stops escalation before it happens.
I wasn't bullied in school, but not for lack of attempts. When they threw a ball of paper at me, I'd throw it back. When they asked if I want to fight, I'd say "ok". This happened many times, and not once did I actually have to fight anyone and therefore never I got suspended. Even the much bigger kid a year ahead of me just didn't show up to the scheduled fight (thankfully).
Even as an adult, the same approach worked well the one time I was in a bizarre enough situation to need it in a literal sense. The only time I ever got in a real fight was when it was too scary to say "ok" and I tried to run away instead.
To give a more "adult" example, if a neighbor in your apartment complex starts yelling at you for "messing with" the jacuzzi heater, the right posture isn't to yell back, it's to just to tell him that you were fixing it, then turn your back on him and go back to enjoying your jacuzzi session until he realizes he's being a dick and apologizes. It's absolutely a useful skill that shows up in places that are worth being in.
That doesn't mean "fight back" won't get you in trouble, just that you have to make sure you're doing the right thing. The posture isn't "Fuck me? No fuck you!", or "I'm not gonna take your shit anymore!", it's "ok". The former can get you in a lot of trouble -- both with bullies and with the school, because it's actively escalating. The latter deflates attempts to bully really quickly because there's just no place to put them.
In hindsight, the one time I failed to avoid a fight it's because I didn't commit to either "ok" or running. I chose to run, and when they gave chase and started kicking at us I decided "I guess I'm gonna have to fight back" -- which they weren't expecting but by then they didn't have a graceful exit. Expectation management.
So yes, "just fight back!" is naive and dumb. And "opportunities to learn to stand up for yourself" are of no use if you don't manage to learn these things. At some point you just gotta pull your kid.
At the same time, there's something real there too. There is an opportunity, if you manage to rise to it. And it applies to adult life as well in far more subtle ways.