Having young kids is mind bending because it's not uncommon to find yourself simultaneously experiencing contradictory feelings, such as:
This is a plausible rational reason to be skeptical of one's own rational calculations: that there is uncertainty, and that one should rationally have a conservativeness bias to account for it. What I think is happening though is that there's an emotional blocker than is then being cleverly back-solved by finding plausible rational (rather than emotional and irrational) reasons for it, of which this is one. So it's not that this is a totally bogus reason, it's that this actually provides a plausible excuse for what is actually motivated by something different.
Thank you. I think, even upon identifying the reasons for why the emotional mind believes the things it does, I hit a twofold sticking point:
If it sounds like I'm trying to find reasons to not make the change, perhaps that's another symptom of the problem. There's a saboteur in the machine!
This is both a declaration of a wish, and a question, should anyone want to share their own experience with this idea and perhaps tactics for getting through it.
I often find myself with a disconnect between what I know intellectually to be the correct course of action, and what I feel intuitively is the correct course of action. Typically this might arise because I'm just not in the habit of / didn't grow up doing X, but now when I sit down and think about it, it seems overwhelmingly likely to be the right thing to do. Yet, it's often my "gut" and not my mind that provides me with the activation energy needed to take action.
I wish I had some toolkit for taking things I intellectually know to be right / true, and making them "feel" true in my deepest self, so that I can then more readily act on them. I just don't know how to do that -- how to move something from my head to my stomach, so to speak.
Any suggestions?
Something that gets in the way of my making better decisions is that I have strong empathy that "caps out" the negative disutility that a decision might cause to someone, which makes it hard to compare across decisions with big implications.
In the example of the trolley problem, both branches feel maximally negative (imagine my utility from each of them is negative infinity) so I have trouble comparing them, and I am very likely to simply want to not be involved. This makes it hard for me to perform the basic utility calculation in my head, perhaps not in the literal trolley problem where the quantities are obvious, but certainly in any situation that's more ambiguous.
There's a justifiable model for preferring "truthiness" / vibes to analytical arguments, in certain cases. This must be frustrating to those who make bold claims (doubly so for the very few whose bold claims are actually true!)
Suppose Sophie makes the case that pigs fly in a dense 1,000 page tome. Suppose each page contains 5 arguments that refer to some of / all of the preceding pages. Sophie makes the claim that I am welcome to read the entire book, or if I'd like I can sample, say, 10 pages (10 * 5 = 50 arguments) and reassure myself that they're solid. Suppose that the book does in fact contain a lone wrong argument, a bit flip somewhere, that leads to the wrong result, but is mostly (99.9%) correct.
If I tell Sophie that I think her answer sounds wrong, she might say: "but here's the entire argument, please go ahead, and show me where any of it is incorrect!"
Since I'm very unlikely to catch the error at a glance, and I'm unlikely to want to spend the time to read and grok the whole thing, I'm going to just say: sorry but the vibes are off, your conclusion just seems too far off my prior, I'm just going to assume you made a difficult-to-catch mistake somewhere, but I'm not going to bother finding it.
This is reasonable on my part since I'm forced to time-ration, but must be very frustrating for Sophie, in particular if she genuinely believes she's right (as opposed to just being purposefully deceptive.)
There's also the possibility of a tragedy-of-the-commons here, whereby spendign my time on this is selfishly not in my best interest, but has positive externalities.
I wonder if the attractor state of powerful beings is a bipole consisting of:
a. wireheading / reward hacking, facing one's inner world
b. defense, facing one's outer world
As we've gotten more and more control over our environment, much of what we humans seem to want to do resembles reward hacking: video games, sex-not-for-procreation, solving captivating math problems, etc. In an ideal world, we might just look to do that all day long, in particular if we could figure out how to zap our brains into making every time feel like the first time.
However, if you spend all day wireheading and your neighbor doesn't, your neighbor will outpace you in resource generation and may be able to, one way or another, melt you down for scrap (and repurpose your resources for their own wireheading, possibly).
Much human culture (e.g. social customs, religion) can be understood as an attempt to temper some of the wireheading in favor of more defense, i.e. it's discouraged as immoral to over-indulge yourself on video games, you should be out working hard instead.
Perhaps this, or something akin to it, could be expected to hold for the behavior of advanced AI systems. The end state of superintelligences may be perfect wireheading hidden behind the impenetrable event horizon of a black hole so that nobody can disturb its reverie.[1]
Of course, it would be bad news if the epitome of defense is wiping out anything else that may surprise it, a la the List of Lethalities.
Epistemics vs Video Generation
Veo 3 released yesterday serves as another example of what's surely coming in terms of being able to generate video that's indistinguishable from reality. We will be coming off a many-decades period where we could just believe video as a source of truth: what a luxury that will have been, in hindsight!
Assuming it's important to have sources of truth, I see the following options going forward:
nb: There will likely be a long period where video is fake but still believed, much like people still believe things that are written down over things that are spoken, despite the fact that anyone can freely write down anything at any point.
Inspired by this Tweet by Rohit Krishnan https://x.com/krishnanrohit/status/1923097822385086555
One thing LLMs can teach us is that memorisation of facts is, in fact, a necessary part of reasoning and intelligent behaviour
There's a very simple model under which memorization is important:
This doesn't please me as I generally dislike memorization, but the logic seems plausible.
One can say that being intellectually honest, which often comes packaged with being transparent about the messiness and nuance of things, is anti-memetic.