One can say that being intellectually honest, which often comes packaged with being transparent about the messiness and nuance of things, is anti-memetic.

Having young kids is mind bending because it's not uncommon to find yourself simultaneously experiencing contradictory feelings, such as:

• I'm really bored and would like to be doing pretty much anything else right now.
• There will likely come a point in my future when I would trade anything, anything to be able to go back in time and re-live an hour of this.

This is a plausible rational reason to be skeptical of one's own rational calculations: that there is uncertainty, and that one should rationally have a conservativeness bias to account for it. What I think is happening though is that there's an emotional blocker than is then being cleverly back-solved by finding plausible rational (rather than emotional and irrational) reasons for it, of which this is one. So it's not that this is a totally bogus reason, it's that this actually provides a plausible excuse for what is actually motivated by something different.

Thank you. I think, even upon identifying the reasons for why the emotional mind believes the things it does, I hit a twofold sticking point:

• I consider the constraints themselves (rarely in isolation but more like the personality milieu that they are enmeshed with) to be part of my identity, and attempting to break them is scary in both a deep existential loss of self sense, and in a "this may well be load bearing in ways I can't fully think through" sense
• Even orthogonal to the first bullet, it's somehow hard to change them even though with my analytical mind I can see what's going on. It's almost like the emotional Bayesian updating has brought these beliefs / tendencies to a very sharp peak long ago, but now circumstances have changed but the peak is too sharp to belief it away with new experience.

If it sounds like I'm trying to find reasons to not make the change, perhaps that's another symptom of the problem. There's a saboteur in the machine!

This is both a declaration of a wish, and a question, should anyone want to share their own experience with this idea and perhaps tactics for getting through it.

I often find myself with a disconnect between what I know intellectually to be the correct course of action, and what I feel intuitively is the correct course of action. Typically this might arise because I'm just not in the habit of / didn't grow up doing X, but now when I sit down and think about it, it seems overwhelmingly likely to be the right thing to do. Yet, it's often my "gut" and not my mind that provides me with the activation energy needed to take action.

I wish I had some toolkit for taking things I intellectually know to be right / true, and making them "feel" true in my deepest self, so that I can then more readily act on them. I just don't know how to do that -- how to move something from my head to my stomach, so to speak.

Any suggestions?

Something that gets in the way of my making better decisions is that I have strong empathy that "caps out" the negative disutility that a decision might cause to someone, which makes it hard to compare across decisions with big implications.

In the example of the trolley problem, both branches feel maximally negative (imagine my utility from each of them is negative infinity) so I have trouble comparing them, and I am very likely to simply want to not be involved. This makes it hard for me to perform the basic utility calculation in my head, perhaps not in the literal trolley problem where the quantities are obvious, but certainly in any situation that's more ambiguous.

There's a justifiable model for preferring "truthiness" / vibes to analytical arguments, in certain cases. This must be frustrating to those who make bold claims (doubly so for the very few whose bold claims are actually true!)

Suppose Sophie makes the case that pigs fly in a dense 1,000 page tome. Suppose each page contains 5 arguments that refer to some of / all of the preceding pages. Sophie makes the claim that I am welcome to read the entire book, or if I'd like I can sample, say, 10 pages (10 * 5 = 50 arguments) and reassure myself that they're solid. Suppose that the book does in fact contain a lone wrong argument, a bit flip somewhere, that leads to the wrong result, but is mostly (99.9%) correct.

If I tell Sophie that I think her answer sounds wrong, she might say: "but here's the entire argument, please go ahead, and show me where any of it is incorrect!"

Since I'm very unlikely to catch the error at a glance, and I'm unlikely to want to spend the time to read and grok the whole thing, I'm going to just say: sorry but the vibes are off, your conclusion just seems too far off my prior, I'm just going to assume you made a difficult-to-catch mistake somewhere, but I'm not going to bother finding it.

This is reasonable on my part since I'm forced to time-ration, but must be very frustrating for Sophie, in particular if she genuinely believes she's right (as opposed to just being purposefully deceptive.)

There's also the possibility of a tragedy-of-the-commons here, whereby spendign my time on this is selfishly not in my best interest, but has positive externalities.

I wonder if the attractor state of powerful beings is a bipole consisting of: 

a. wireheading / reward hacking, facing one's inner world 
b. defense, facing one's outer world

As we've gotten more and more control over our environment, much of what we humans seem to want to do resembles reward hacking: video games, sex-not-for-procreation, solving captivating math problems, etc. In an ideal world, we might just look to do that all day long, in particular if we could figure out how to zap our brains into making every time feel like the first time.

However, if you spend all day wireheading and your neighbor doesn't, your neighbor will outpace you in resource generation and may be able to, one way or another, melt you down for scrap (and repurpose your resources for their own wireheading, possibly).

Much human culture (e.g. social customs, religion) can be understood as an attempt to temper some of the wireheading in favor of more defense, i.e. it's discouraged as immoral to over-indulge yourself on video games, you should be out working hard instead.

Perhaps this, or something akin to it, could be expected to hold for the behavior of advanced AI systems. The end state of superintelligences may be perfect wireheading hidden behind the impenetrable event horizon of a black hole so that nobody can disturb its reverie.^[1]

1. ^{^}

   Of course, it would be bad news if the epitome of defense is wiping out anything else that may surprise it, a la the List of Lethalities.

Epistemics vs Video Generation

Veo 3 released yesterday serves as another example of what's surely coming in terms of being able to generate video that's indistinguishable from reality. We will be coming off a many-decades period where we could just believe video as a source of truth: what a luxury that will have been, in hindsight!

Assuming it's important to have sources of truth, I see the following options going forward:

1. we will have to just accept that our world has permanently become epistemically worse. sad!
2. we will have to trust real life more than we do external evidence. this seems very limiting: my real life friends don't have much insight into what's happening in Ukraine, for example
3. we will need to pair video with a sort of "chain of custody" whereby the video itself will not be worth anything (epistemically) unless it comes with someone, whose identity is established in some other way, vouching for it being true. this is, in a way, how we presently treat written evidence.
4. lacking a trusted source, but making up for it by combining multiple videos of the same thing. imagine multiple angles of the same event (as confirmed by hard-to-spoof video metadata) corroborating each other (perhaps this can even be AI-enabled, in a privacy-preserving way). hoping that this is easy to verify but hard to fake (hard to generate mutually consistent multiple angles of the same thing) though this might be wishful thinking since this seems like something that can be trained eventually, but maybe buys us a few years
5. the physics solution: from first principles, video had the property that it was much easier to verify than it was to fake (a la certain encryption). video is, reductively, just a time series of photons. perhaps we can build The Truth Camera, which captures lots of other particles (or maybe captures photons at a frame rate and resolution that is costly, and thus there isn't much data of, and thus is hard to credibly fake). essentially perhaps we can snapshot more aspects of reality in such a way that we re-create the condition that made video credible: easy to verify, hard to generate.

nb: There will likely be a long period where video is fake but still believed, much like people still believe things that are written down over things that are spoken, despite the fact that anyone can freely write down anything at any point.

Inspired by this Tweet by Rohit Krishnan https://x.com/krishnanrohit/status/1923097822385086555

One thing LLMs can teach us is that memorisation of facts is, in fact, a necessary part of reasoning and intelligent behaviour

There's a very simple model under which memorization is important:

• if good reasoning resembles test-time compute, judgement involves iterating through lots of potential hypotheses and validating them
• the speed with which you validate each hypothesis is partly a function of the time it takes to fetch the data necessary to assess it
• when that data is memorized (i.e. stored "in your brain's weights") access is really fast
• when that data is stored externally (i.e. have to run a google search to find it) validation is OOMs slower
• so you'll be OOMs slower if you haven't internalized the context of the problem
• further, there are upper bounds on the slower process (because you get bored, or you fatigue, etc.) so this difference in speed may also end up being a difference in capability. it's not just that it's OOMs slower, it may not even get there

This doesn't please me as I generally dislike memorization, but the logic seems plausible.