Whether an AI feels emotions depends on how loose you are with the category "emotion". Take the emotion of curiosity. Investigating the environment is sometimes beneficial, due to the value of information. Because this behavior is quite complex, and the payoff is rare and indirect, reinforcement learners will struggle to learn it by default. However, there was a substantial evolutionary pressure towards minds that would display curious behavior. Evolution, being the blind idiot god, built a few heuristics for value of information that were effective in the environment of evolutionary adaptation, and hard wired these to the pleasure center.
In the modern environment, curiosity takes on a life of its own, and is no longer a good indication of value of information. Curiosity is a lost purpose.
Does AIXI display curiosity? It's calculating the value of information exactly. It will do a science experiment if and only if the expected usefullness of the data generated is greater than the expected cost of the experiment.
This is a meaningless semantic question, AIXI displays behavior that has some similarities to curiosity, and many differences.
I expect a from first principles AI, MIRI style, to have about as much emotion as AIXI. A bodge it till you make it AI could have something a bit closer to emotions. The Neural net bashers have put huristics that correlate to value of information into their reinforcement learners. An evolutionary algorithm might produce something like emotions, but probably a different set of emotions than the ones humans feel. An uploaded mind would have our emotions, as would a sufficiently neuromorphic AI.
I do sort of expect AI to have something like emotion, because I think since humans in general tend to focus on the subjective aspect of emotions to the exclusion of their functional roles. I think of emotions as something like "brain modes" designed to reconfigure our limited mental resources to make certain behaviors more likely and other behaviors less likely. Given that AI will also be similarly bounded, I would expect to see something similar allowing the AI to alter its thinking in ways that would feel to the AI subjectively similar to our own experience of emotions and would carry out a similar role of temporarily reconfiguring the AI to be better optimized for particular scenarios.
What those specific emotions would be seems like a thing you could speculate about and *might* be fun and interesting to try but mostly because I think answers will provide hermeneutical fodder for investigating our (possibly confused) beliefs about emotions and AI.
Ehn, this is very hard to predict, because it'll depend a whole lot on whether AIs are evolved, created, or copied from humans (or rather, what mix of those mechanisms is used to bootstrap and improve the AIs). To the extent that AIs learn from or are coerced to human "values", they'll likely encode all human emotions. To the extent that they're evolved separately, they'll likely have different, possibly-overlapping sets.
As you hint at, human emotions are evolved strategies for decision-making. Since an AI is likely to be complex enough that it can't perfectly introspect itself, and it's likely that at least many of the same drives for expansion, cooperation, and survival in the face of competitive and cooperative organisms/agents will exist, it seems reasonable to assume the same emotions will emerge.
One idea I have about emotions is that they are evolutionary adaptations which evolved before "reasoning intelligence" (whatever it is) in mammals and their goal was to turn on specific mode of action. For example, anger turns on the mode of action of fight response, which includes higher blood pressure, lower sensitiveness to pain, and making fists. Most of the basic emotions could be explained as modes of action (e.g. fear, sexual arousal), but it is not the full story.
If we read any book on ethology of, say, birds, we found that an important part of their behaviour is demonstrations. A cat is not only ready to fight, but the cat demonstrates its readiness to fight to the opponent in credible way by raising its hair and vocalization. I read that, in the case of birds, demonstrations are more important than actual fights, as demonstrations better show who won and who lost - without physical damage to both sides.
Humans brain is built upon animal brain, so it inherited most of animal features, but in suppressed and more flexible form. In animals, emotions work as a rule-based system which control behavior and signaling. This rule system is relatively simple and "mechanical", so there is nothing mystical in emotions or difficult for reproduction in AI. (Like in case of a cat: if you see small animal - hunt; if you see animal of your size - fight and demonstrate; if you see the animal much large than you - run). Also, there is nothing "human" in emotions - they are an animal part of us.
Emotions also sometimes can give us quicker estimation of the nature of a situation then reasoning in System 2, as they result from quick estimation of a situation by a large neural net, which could peek up many subtle clues and presents them as one conclusion, like, "Run!"
Given all this, emotions as demonstrations of a chosen mode of action may be used by AI - the same way as national states demonstrate their military posture. They could be also used for quick estimation of a new situation for risks by a artificial neural net trained on such situations.
[this post is completely fluffy and superficial]
When talking about AIs, the issue of emotions often comes up. I normally finesse the question by saying that we don’t really know if AIs would have things we would recognise and emotions, and would you stop pestering me, I have some serious and scary points to make.
But… If we take the question seriously for a second, and allow ourselves to speculate… The main angle of attack is to wonder: what are emotions for, what functions do they serve? And would they serve the same functions in an AI?
So, here’s a speculative cross-section of emotions, and whether AIs are likely to have versions of them:
At their simplest, joy and sadness are responses to the situation turning out better or worse than expected. These are essential to learning, and the AI would certainly have some version of them. Similarly, boredom is a sign that the AI should do more exploration of its environment, empathy is useful for understanding other agents, and interest is a sign that a certain area should be investigated in priority.
If the AI is able to credibly pre-commit to certain actions (eg if attacked, it will retaliate), then some of these emotions won’t be needed. But it might not be able to credibly signal that, in which case anger would be useful: other agents will be less inclined to exploit it, lest it react aggressively (note that giving in to anger is generally a disadvantage, but being known to give in to anger is often an advantage). Kindness and cruelty are similarly tools for managing social interactions; if the AI is transparent and can pre-commit, they won’t be needed.
Empathy is the key AI emotion, but it’s possible that it may award special categories for other agents with especially positive or negative traits. Surprise, finally, is a signal that unlikely events are happening, and a re-assessment is needed; if the AI cannot implement its assessments and reassessments in a smooth way, it may need some threshold of surprise to trigger a reaction.
These emotions are all connected with protecting our own physical body, so an AI would only feel them if it had a physical body itself, and couldn’t just think through the issues dispassionately.
These are very specific to the human condition, and the human social circumstances, so they are unlikely to be felt by generic AIs. Now, it's always possible for AIs to feel these, but there's no necessary reason for them to feel them, as far as we known now.
Finally, we should consider that AI’s could have emotions that we never had, because humans were never in circumstances where we would use them. What do you think these might be like? I’ll present two speculative ideas: first of all, AIs should have an intuitive grasp of statistics and probability that is much better than our own, so they may have statistical instincts. Secondly, if the AI gets copied a lot, self-loyalty – loyalty to copies and slightly-divergent-copies – may become useful, and instinctive ways of resolving debates with itself may become common.