This post is a response to Reward is not the optimization target, which someone recommended I read after I made a post on my blog.

People have grown skeptical of the claim that an artificial intelligence would learn to intrinsically value reward over what the reward is tied to. I would have to disagree. I am going to argue that any AI that would surpass its designers would have to value reward over it's goals, otherwise it wouldn't be smarter than its designers.

Being smarter than its designers is defined as coming up with solutions that the designers couldn't.

I am going to make a comparison of computers to humans and evolution. I will be talking about evolution as if it had volition, but that's just a metaphor to get my point across easier. You could replace it with God if you want, I am just describing the way our world works.

Evolution "wants" us to spread our genes. There are two ways it could do that. It could have all the instructions of how to reproduce hard coded into our DNA, or it could push us into reproducing by rewarding certain behavior ~ not dying, having sex, and letting us figure out the rest. Although you can find the first method in simple viruses and bacteria, the second has led to more complex organisms completing the "goal" of passing on their genes since the beginning of life. However, recently, there has been one species that has been messing up this system. Humans.

Humans have found ways of achieving reward without accomplishing "evolution's goals". Some people eat lots of ice cream because it's rewarding to them, but excessive ice cream eating leads to you dying, not the opposite. Some people have sex with condoms, leading to no children being created. Drugs have no counterpart in nature, but they are widespread in society. In a sense, we have "outsmarted" evolution, using it's processes for our own goals rather than what they are "supposed" to do. A system meant to promote reproduction in humans can lead to us preventing it.

If you look at the history of early artificial intelligence research, most of it was trying to hard code an algorithm to accomplish whatever goals the researcher wanted. This kind of research has accomplished many things, Optical Character Recognition, chess playing robots, and even basic chat bots. Many of these things were considered artificial intelligence in the past, and some people thought you would need...

Posts

Wikitag Contributions

Comments