Rationality versus Short Term Selves



Many of us are familiar with the marshmallow test.If you are not, here.

It is predictive of success, income, level of education, and several other correlated measures.

I'm here to argue for the marshmallow eaters, as a devil's advocate. Contra Ainslie, for instance. I do it out of genuine curiosity, real suspicion, and maybe so that smart people get me back to my original position, pro-long term.

There is also the e-marshmallow test (link is not very relevant), in which children have to face the tough choice between surfing an open connected computer with games, internet etc... and waiting patiently for the experimenter to get back. Upon the experimenter's arrival, they get a pile of marshmallows. I presume it also correlates with interesting things, though haven't found much on it.

I have noticed that rationalists, LessWrongers, Effective Altruists, Singularitarians, Immortalists, X-risk worried folk, transhumanists, are all in favor of taking the long view.  Nick Bostrom starts his TED by saying: "I've been asked to take the long view"

I haven't read most of Less Wrong, but did read the sequences, the 50 top scoring posts and random posts. The overwhelming majority view is that the long view is the most rational view. The long term perspective is the rational way for agents to act.

Lukeprog, for instance, commented:

"[B]ut imagine what one of them could do if such a thing existed: a real agent with the power to reliably do things it believed would fulfill its desires. It could change its diet, work out each morning, and maximize its health and physical attractiveness."

To which I responded:

I fear that in this phrases lies one of the big issues I have with the rationalist people I've met thus far. Why would there be a "one" agent, with "its" desires, that would be fulfilled. Agents are composed of different time-spans. Some time-spans do not desire to diet. Others do (all above some amount of time). Who is to say that the "agent" is the set that would be benefited by those acts, not the set that would be harmed by it.
My view is that picoeconomics is just half the story.
In this video, I talk about picoeconomics from 7:00 to 13:20 I'd suggest to take a look at what I say at 13:20-18:00 and 20:35-23:55, a pyramidal structure of selfs, or agents. 

So you don't have to see the video, let us design a structure of selfhood.

First there is intertemporal conflict, conflict between desires that can be fulfilled at different moments of time. Those reliably fall under a hyperbolic characterization, and the theory that described this is called Picoeconomics, mostly developed by George Ainslie in his Breakdown of Will and elsewhere.

But there is also time-length, or time-span conflict.  The conflict that arises from the fact that you are, at the same time, the entity that will last 200milliseconds, the entity that will last one second, and the entity that will last a year, or maybe, a thousand years.

What do we (humanity) know about personal identity at this point in history? If mainstream anglophone philosophical thought is to be trusted, we have to look for Derek Parfit's work Reasons and Persons, and posterior related work, to get that.

I'll sum it up very briefly: As far as we are concerned, there are facts about continuity of different mental classes. There is continuity of memory, continuity of conscious experience, continuity of psychological traits and tendencies, continuity of character, and continuity of inferential structure (the structure that we use to infer things from beliefs we acquire or access).   

For each of these traits, you can take an individual at two points in time and measure how related It1 and It2 are with respect to that psychological characteristic.  This is how much I at T2 is like himself at T1.

Assign weights for traits according to how much you care (or how important each is in the problem at hand) and you get a composed individual, for which you can do the same exercise, using all of them at once and getting a number between 0 and 1, or a percentage. I'll call this number Self-Relatedness, following the footsteps of David Lewis.

This is our current state of knowledge on Personal Identity: There is Trait-Relatedness, and there is Self-Relatedness. After you know all about those two, there is no extra fact about personal identity. Personal Identity is a confused concept, and when we decompose it into less confused, but more useful, sub-sets, there is nothing left to be the meta-thing "Personal Identity".

Back to the time-length issue, consider how much more me  the shorter term selves are (that is how much more Self-Relatedness there is between any two moments within them). 

Sure if you go all the way down to 10 milliseconds, this stops being true, because there are not even traits to be found. Yet, it seems straightforward that I'm more like me 10 seconds ago than like me 4 months ago, not always, but in the vast majority of cases.

So when we speak of maximizing my utility function, if we overlook what me is made of, we might end up stretching ourselves to as long-term as we possibly can, and letting go of the most instantaneous parts, which de facto are more ourselves than those ones.

One person I met from the LessWrong Singinst cluster claimed: "I see most of my expected utility after the singularity, thus I spend my willpower entirely in increasing the likelihood of a positive singularity, and care little about my current pre-singularity emotions"

Is this an amazing feat of self-control, a proof that we can hope to live according to ideal utility functions after all? Or is it a defunct conception of what a Self is?

I'm not here to suggest a canonical curve of time-lengths of which the Self is composed. Different people are different in this regard. Some time-lengths are stretchable, some can be shortened. Different people will also value the time-lengths differently. 

It would be unreasonable for me to expect that people would, from now on, put on a disclaimer on their writings "I'm assuming 'rational' to mean 'rational to time-lenghts above the X treshold' for this writing". It does, however, seem reasonable to keep an internal reminder when we reason about life choices, decisions, and writings, that not only there are the selves which are praised by the Rationalist cluster, the long term ones, but also, the short term ones.

A decision to eat the marshmallow can, after all, be described as a rational decision, it all depends on how you frame the agent, the child.

So when a superintelligence arises that, despite being Friendly and having the correct goals, does the AGI equivalent of scrolling 9gag, eating Pringles and drinking booze all day long, tell the programmers that the concept of Self, Personal Identity, Agent, or Me-ness was not sufficiently well described, and vit cares too much for vits short-term selves. If they tell you: "Too late, vit is a Singleton already" you just say "Don't worry, just make sure the change is ve-e-e-ery slow..."