Main points

The notion of constant progress of time applies both to AI and humans. However, in decision theories and AI research, it makes things more complicated and seem to be frequently disregarded for simplicity.
An evolving AI may face the need to predict the decision-making of its future version, in absence of information about future.
Time also may has something to do with human utility and qualia, along which AGI needs to be aligned - and we are in a urgent need of better understanding.
Futures in which AGI is designed to maximize human utility without better understanding of it seem to have a lot of people getting wireheaded.

I'm not discussing worst-case AGI scenarios here. I tried to imagine the best case scenario under some conditions, but failed.

In 'Terms and assumptions' section down below I tried, though also failed probably, to clarify things I assumed.

John's timeless self-delusion

I don't exactly remember anymore why I wrote this once, but I'll use this story to illustrate this post.

Suppose John wants to murder Bill, and has to escape arrest by the thought police that will come and check on him tomorrow morning. Luckily or not, John is also a meditation expert and can modify his perceptions and wishes. John could've dropped his intent towards Bill if he wanted to. However, to fulfill his desire at the time and make sure Bill gets killed, he chooses instead to meditate to a state of insanity, in which he believes that Bill is a superhuman that loves playing the surprise catch-the-brick-with-your-head game at mornings, and was his best friend all along. Tomorrow, after passing thought police's routine check, John proceeds to drop a brick on poor Bill's head from a roof, then finds himself confused and unhappy ever since.

Originally, the point of this story wasn't that John is a poorly-designed runaway AI that deludes itself, then does bad things. John was intended to be human, and a subject to experience and own qualia. (Although one arguably doesn't exclude another.) The point probably was to demonstrate it's difficult to describe decision-making based on preferences, when preferences change over time.

I found myself at lack of dictionary for distinguishing between future and past selves, describing various degrees of cooperation between them, and talking about subjective utility. I ...

Posts

Wikitag Contributions

Comments