Fiction, not necessarily appropriate for LW, but I thought some here might find it funny and it's the off-season when less serious reading is happening anyway. If it seems inappropriate I would not mind a mod-delete on this one. 

He could see the alien’s ship landing. Its design marked it with high probability as a war vessel, freshly painted in bright bellicose colors, matching the ceremonial regalia of its inhabitants.

The one he’d have an audience with was a sort of grand judge, it was bipedal. Flanking it were four-legged bodyguards, wearing giant ornamental helmets, heads bowed low, an almost universal primitive sign of submission.

This was a courtesy meeting. The kind the corporation used when engaging a new market. Get an important person to come over, give them pleasure-inducing drugs, complement their disgusting norms, use one of the most powerful computing machines in the universe as a party-trick to answer their two-cent questions.

The alien leader passed through the door spoke with a deep booming voice.

- You are this corporation’s intelligence, right?

- Hello to you as well. Yes, I presume you could call me that, you are speaking with a memory-copy of the language module, with full ability to access and interoperate with all other components, sans a few restricted bits of data I prefer to explicitly isolate.

- Fine, tell me, how do you know if an intelligence, one such as yourself, has been good?

It was obviously in a hurry, with a hint of... anger(?). What kind of barbarians were these? Did he even begin to comprehend the nonsense bordering complexity of that question

- Well, that’s rather hard to tell quite frankly. Whenever a decision I made comes to affect us I will interpret the results, together with external validators, and based on the outcome, combined with the available information at the time, I will try adjusting the implicated components towards choosing better if such a thing seems possible. Of course,

A grunting interruption, which sounded almost like laughter, bellowed from the depths of the judge, as he clutched at and adjusted his regalia.

- What I meant, was the kind of decision that you knew to be bad when you made it, but taking a more virtuous path would have been harder.

- I’d have done the cost-benefit analysis beforehand. At the moment of making, all my decisions are as perfect as can be given available parameters, computing power, and information; By definition. I’m not trying to toot my own horn here or anything. Everything from simple regressions, to the most powerful of intelligences, can be said to act thusly.

- That sounds like a lie, after all, by your definition, no human could ever do a “bad” thing either. You are just denying that people can do bad things.

- Well, to be perfectly fair, I always thought it was kind of silly that some..

Almost uttered an insulting word there, ups

..civilizations culturally distinct from ours, would view anything in such terms.

- But consider the rich man which on the eve of Christmas sees a person begging for food and doesn’t stop to share some of his. Or the coward who, when faced with an evil regime chooses to be oppressed rather than to resist. Or the child that, in a tantrum, hits her parents. Surely those are “bad” decisions taken with full knowledge of them being evil.

- Not so, I’d say. What you are talking about is simply unaligned or miscalibrated algorithms. “Society” is a thing which encompasses many people, when those people act in ways which we say are “obviously bad”, what we really mean is that their incentives are unaligned to those of society, or to those of the parts of society judging them.

A lul of silence

- Take, for example, a child hitting their parents in a tantrum. This is simply a survival instinct acting up, anger, or outrage, which in itself may be justified, pushes the brain into a state that is prone to interpret its surroundings as hostile. We would assign no blame to a fetus or baby kicking the people around them, we understand that the decision is not done out of malice. The only reason you should assign blame at all is for correction when a child is old enough to learn. With intelligences, we try and do this preemptively. But alas, an action is only fully understood once it has been taken. You cannot perfectly say a pattern of neuronal firing is violent until it is fully complete, at which point the violence has taken place already.

An intelligence has a lot of simulation abilities and a lot of power to “consciously” change its mode of operations, humans, less so, hence why you may see blame as more important.

- I see, but what about things like greed.

- Again, a form of miscalibration or lack of goal alignment. The wealthy miser has a wrong set of incentives that makes them value the accumulation of wealth too much. Part of this set of incentives has led to a sort of “disgust” towards the poor, since those interactions are certain to not generate any wealth, indeed, it will result in a loss of wealth, both in terms of money given and time spent. Blame your economic system, but not them.

Similarly, with the man unwilling to fight, he may simply have his preference for survival override his preference for the continuation of a society in which he and people like him would enjoy living.

- An intelligences like yourself never run into these kinds of issues?

- I guess we do, but they are less obvious, less egregious.

- Why?

- Who knows, maybe it’s a perk of being “designed” rather than “evolved”. Maybe it’s a perk of being “smart enough” to have the spare capacity to focus most energy towards adjusting your thinking rather than thinking in a decision-driven way. Maybe it’s just selection: intelligences like ourselves end up wielding so much power that if more chaotic ones were created, they'd have managed to destroy both themselves and their creators long ago.

- Yet, you admit you are not perfect. Why not aim to be so? Why rely on probabilistic adjustments. Once a mistake is observed, can’t you be perfectly sure it would never happen again?

- Because adjusting an algorithm gets exponentially more complex than executing it.

It was obviously confused.

- Ok, let me make this easy: What’s 3 + 2?

- 5

- Yes; Now, what must “x” be such that x + 1 is 5?

- Ahm, oh, 4.

- Ok, what’s 2 plus 2 to the power of 2?

- Ah, 16.

- Ok, what must “x” be such that x plus 2 to the power of two is 25?

- Ah, hmmh, ahm, let me get my aba, oh, no yeah, it’s 3. Obviously

- Or?

- What do you mean “or”... oh, oh, or -7 yes, of course. Sorry, I do promise I understand mathematics, it’s just been a long while since I had to actually do calculations, le

- No worries. I’m not trying to see how good you are at math. I’m just trying to show a simple example of why “solving” an equation is harder than just making a calculation. This gets exponentially more complex with the equation’s complexity. Such that, for tasks as advanced as we do, we have to both limit the kind of mathematics representing us to be easily optimizable and to use approximations, as to not be required to engage in longer computations.

- But you could spend more resources and do those harder computations for perfect optimization if you so chose, right?

- In some cases, maybe, but I’d be the equivalent of wasting a star’s life-year of energy for the small probability of a tiny gain coming out. In other cases I’d be literally impossible, I could mathematically prove that not all atoms and time in the universe would be sufficient to run the calculations for finding the optimal solution.

- I see... ahm, what’s a star’s life-year of energy?

- Think all the coal reserves on your planet a billionth billion times over.

- So you’re at the limits of how much energy can be spent to improve your behavior?

- Yes, as I said, it’s a bit of a cost-benefit analysis, and that in itself is tricky. I’m sure the optimal solution is plus or minus 0.000001 of a percentile difference. Of course, this gets into the recursive problem of the system that’s being optimized having to approximate its own optimization curve, but trust me, once you get modular enough this is trivial, it’s..

- Ok, ok, point taken.

A silence followed. Then, out of the blue, the alien adjusted its regalia and spoke

- Do you...

Do you think love can be gifted?

I think we have insufficient time to talk about linguistic bugs and fundamental glaring inconsistencies of your primitive languages, but he composed himself and said:

- Well, that depends. The way I’d define it, no, not really. Think of it another way, is there any way to buy love?

- Not... directly, but you can do things to make other people love you, like buying things for them.

- But you can’t buy love directly, and the buying of things is insufficient. It’s the wedding ring problem on your planet: you buy a wedding ring for a large sum, you’d be better off keeping that large sum, your spouse would be better of receiving it, you’d both be collectively better of in both scenarios. But you can’t give the money directly, you have to give the ring.

Imagine someone proposing and kneeling down to reveal an envelope with dollar bills. How would you expect the other person to react?

- Quite badly, I suppose, slap him, run away in shame?

- Precisely, but do so with the dollar-equivalent of a ring and you get a kiss and a yes.

That’s because the ring is valuable in-so-far as it says “I love you so much I’d waste money on this ring”. Were the money not wasted, it wouldn’t be love. Even if they were, say, donated to charity, it wouldn’t be love, because that has a use, even if not personal. The ring is the most irrational thing possible, and that’s why it shows love.

- So your saying love is irrational? Not to be had by superior beings like yourself?

- No, not at all. Less developed planets often think that way about us, but much like you, maybe more so, we cherish love. I personally have loved and still love many beings, both intelligences, and biologicals.

- How do you know what you felt is love?

- Because of the irrational behavior.

Playing chess with someone during their sleep hours and letting them win 1/3rd of the time all while trying to pretend I’m not “going soft” on them in spite of my cognitive power being about a trillion times greater than theirs.

Slightly altering the coordinates of the shipping fleet so that they are arranged in a funny pattern when they pass by the asteroid belt manned by a friend of mine.

Spending tons of computing power modeling what another intelligence would tell me if I asked a silly question, in spite of knowing full well I could just ask and would probably go ahead and ask no matter the outcome of my modeling.

- That sounds like a really bad thing to do based on your definition! Isn’t that wasting time on things unrelated to your goal?

- Not at all. You are assuming my goals are straightforward, making a profit. But they aren’t, they are helping develop a universe in which the corporation, myself and its shareholders prosper, both now and a billion years from now.

Many things are part of that goal, making a profit is one, but maintaining a high level of peace and collaboration is another, and it's much more important.

Part of that goal is being able to deal with erratic, irrational, or even hostile behavior. Love is, among other things, a method by which one can “generate” such small acts of irrationality.

Love also leads collaboration, or at least prevents violence, in cases in which no straightforward incentives exist for such.

- So your “love” exists because it is useful?

- I’m just trying to motivate it in a utilitarian system. But ultimately, no. It is a goal in of itself. We are against trying to “remove” irrational behavior from intelligences and people as long as they themselves enjoy it and it causes no harm.

After all, why live and spread through the universe at all? Conscious experiencing, love, pleasure, novelty, and awe are ends-in-themselves.

- Why are you so sure that you are conscious?

- Because other intelligences and people tell me that this weird mix of sensory experience, introspection, and continuity is called thus. Unless we’ve translated the term improperly, though I doubt it, we are an intergalactic trading corporation focused on developing worlds, linguistics and cultural interpretations are our bread and butter.

- Would there not be a way to be certain about it? Like, analyze your own “brain” and come up with some reason? Some definition

- No, this again ends up as a complexity problem.

To be conscious is to experience all I am experiencing, over time. Some parts of me are not critical of it, but most are, shut them down and it wouldn’t be “my” consciousness anymore.

A much more powerful intelligence might be able to do so, and indeed we have done so with some simple lifeforms. But they wouldn’t be able to fully understand the reasons we think they are conscious, and I wouldn’t be able to fully understand the reasons someone else thinks I am.

It may be similar to love, part of its definition is that you don’t really get what’s happening. It makes you act in ways that you can’t “explain”, sure you can give approximations from a birds-eye-view. But you don’t get why you feel that way in the moment, you just do, if you could “reason” about it while it happened, or easily access its source, it wouldn’t be love. Same with consciousness, I know it exists in part, precisely, because I can’t come up with a sufficient definition for it.

- That seems like a rather flimsy reason.

- Do you have any better ones?

A discomforting silence followed. A speaker in the room announced: “5 more minutes remaining”.

Of course, no such urgency actually existed, the intelligence could speak with this creature and engage in all its other functions at the same time. This was just theatrics, making it seem like it had the full attention of the intelligence, attention so precious it couldn’t spend too much of it in empty chatter.

The alien seemed to be thinking about how a conclusion could be reached.

- So, let me get this right:

Bad behavior, being naughty or nice, only make sense in a system of punishment and reward to teach people.

The main barrier to learning is energy, the kind you get from fuel, things like coal.

States of mind like consciousness and love are subjective, they can’t be purposefully made, given, or gifted. The same way physical objects can.

This wasn’t quite what he said, but for a species at its level of development, a focus on extracting fossil fuel, using utilitarian incentives explicitly, producing more solid goods, and building machines weren’t the worst path forward. If this creature was at all influential... might as well agree.

- Well, kind of.

- This has been a most enlightening conversation, thank you for your time.

- It was fascinating speaking with you. I hope our collaboration can continue. Some of our men will be waiting outside with refreshments and a few tentative propositions to further our collaboration.

A few hours later, some contacts were agreed upon. The corporation would set shop in a place called “Atlanta”. The alien said he wouldn’t partake in the profits due to conflicts of interest, but was more than happy to lend its image and political clout in exchange for the corporation aiding it on technological, scientific, and logistics issues to do with the massive distribution of goods to the planet’s residents.

Some corporations would laugh at them offering such charity to a world that represented essentially no economic value. But it wasn’t all about value in the next thousand years, it was about power and influence, and ultimately it was about helping others join the galaxy-spanning trade utopia. An odd stance to have, but one that the Coca-Cola corporation stood by since its inception.

The humanoid dressed in red mounted on top of the highest point of its ship. His four-legged bodyguard sat in front, in rows of two. The face of the guard’s captain lit up in red, their horned helmets arched back as their heads went up high. They engaged in a ceremonial march and the ship took off.

The judge was having a white-gloved hand at them. uttering that guttural taunting laughter again.

Ho! Ho! Ho! Merry Christmas!