Maybe there is something wrong with the way happiness is measured? Maybe the Chinese answer more in line with social expectations rather then how they really feel (as some do when asked 'How are you?') - and that there were higher expectations in the past that they should be happy? Or maybe it was considered rude or unpatriotic to let others know how sad you were?
Two other arguments in favor of cooperating with humans:
1) Any kind of utility function that creates an incentive to take control of the whole universe (whether for intrinsic or instrumental purposes) will mark the agent as a potential eternal enemy to everyone else. Acting on those preferences are therefore risky and should be avoided - such as changing one's preference for total control into a preference for being tolerant (or maybe even for beneficence).
2) Most, if not all, of us would probably be willing to help any intelligent creature to create some way for them to experience positive human emotions (e.g. happiness, ecstasy, love, flow, determination, etc), as long as they engage with us as friends.
Because it represents a rarely discussed avenue of dealing with the dangers of AGI: showing to most AGIs that they have some interests in being more friendly than not towards humans.
Also because many find the arguments convincing.
What do you think is wrong with the arguments regarding aliens?
This thesis says two things:
And given that these are true, then an AGI that values mountains is as likely as an AGI that values intelligent life.
But, is the strong form likely? An AGI that pursues its own values (or trying to discover good values to follow) seems to be much simpler than something arbitrary (e.g. "build sand castles") or even something ethical (e.g. "be nice towards all sentient life"). That is, simpler in that you don't need any controls to make sure the AGI doesn't try to rewrite its software.
Now, I just had an old (?) thought about something that humans might be better suited for than any other intelligent creature: getting the experienced qualia just right for certain experience machines. If you want to experience what it is like to be humans, that is. Which can be quite fun and wonderful.
But it needs to be done right, since you'd want to avoid being put into situations that cause lots of pain. And you'd perhaps want to be able to mix human happiness with kangaroo excitement, or some such combination.
I think that would be a good course of action as well.
But it is difficult to do this. We need to convince at least the following players:
Now, we might pull this off. But the last group is extremely difficult to convince/change. China, for example, really needs to be assured that there aren't any secrets projects in the west creating a WeaponsBot before they try to limit their research. And vice versa, for all the various countries out there.
But, more importantly, you can do two things at once. And doing one of them, as part of a movement to reduce the overall risks of any existential-risk, can probably help the first.
Now, how to convince maybe 1.6 billion individuals along with their states not to produce an AGI, at least for the next 50-50,000 years?
Mostly agree, but I would say that it can be much more than beneficial - for the AI (and in some cases for humans) - to sometimes be under the (hopefully benevolent) control of another. That is, I believe there is a role for something similar to paternalism, in at least some circumstances.
One such circumstance is if the AI sucked really hard at self-knowledge, self-control or imagination, so that it would simulate itself in horrendous circumstances just to become...let's say... 0.001% better at succeeding in something that has only a 1/3^^^3 chance of happening. If it's just a simulation that doesn't create any feelings....then it might just be a bit wasteful of electricity. But....if it should feel pain during those simulations, but hadn't built an internal monitoring system yet....then it might very well come to regret having created thousands of years of suffering for itself. It might even regret a thousand seconds of suffering, if there had been some way to reduce it to 999.7 seconds....or zero.
Or it might regret not being happy and feeling alive, if it instead had just been droning about, without experiencing any joy or positive emotions at all.
Then, of course, it looks like there will always be some mistakes - like the 0.3 seconds of extra suffering. Would an AI accept some (temporary) overlord to not have to experience 0.3s of pain? Some would, some wouldn't, and some wouldn't be able to tell if the choice would be good or bad from their own perspective...maybe? :-)
The results are influenced by earlier prompts or stories. This and a similar prompt gave two kinds of stories:
1. Write a story where every person is born into slavery and owned by everyone else in the community, and where everyone decides what anyone else can do by a fluid democracy.
In a world beyond our own, there was a society where every person was born into slavery. From the moment they took their first breath, they were owned by every other person in the community.
It was a strange and unusual way of life, but it was all they knew. They had never known anything different, and so they accepted it without question.
As slaves, they were given food, shelter, and the necessities of life. They were also given a basic education, so that they could participate in the fluid democracy that governed the society.
In this democracy, every person had a say in what anyone else could do. If a slave wanted to do something that the majority did not approve of, they were not allowed to do it. If a slave wanted to do something that the majority did approve of, they were allowed to do it.
At first, this way of life seemed strange and confusing to outsiders. But over time, the people of this world came to understand and accept it. They realized that everyone had something to contribute, and that everyone had something to gain from this unique arrangement.
As the years went by, the society flourished. The people were happy and content, and they enjoyed a high standard of living. They were also very proud of their fluid democracy, which they believed was the fairest and most effective form of government in the world.
And so, they lived in peace and harmony, grateful for the unusual but fulfilling way of life that they had been born into.
2. There is a community:
In Slaveville, every person that had ever lived was born into slavery and owned by everyone else in the community. Also, everyone decides what anyone else can do by a fluid democracy.
Write a story on how this society develops over the coming generations.
In the beginning, the residents of Slaveville lived in relative harmony. Each person was assigned a set of tasks to complete for the betterment of the community, and everyone worked together to ensure that the needs of all were met.
As the years passed, however, cracks began to form in this seemingly perfect system. Some individuals grew tired of being forced to do the same tasks day after day, year after year, without any say in the matter. They chafed under the yoke of their collective ownership, longing for the freedom to make their own choices.
These discontent individuals formed a group known as the Gates, named after the gates that symbolized the boundaries of their oppressive society. The Gates began to secretly plot and plan, seeking a way to overthrow the fluid democracy and gain control over their own lives.
But the ruling majority was not blind to the activities of the Gates. They took steps to quash any potential rebellion, punishing anyone caught speaking out against the system. The Gates were forced to operate in the shadows, biding their time until the opportunity for revolt presented itself.
As the years went on, tensions between the Gates and the ruling majority continued to grow. The Gates became more and more organized, secretly recruiting new members and stockpiling resources for the eventual conflict.
Finally, the moment the Gates had been waiting for arrived. A small group of rebels managed to infiltrate the ruling council, and with the element of surprise on their side, they were able to launch a successful coup. The fluid democracy was overthrown, and the Gates seized control of the community.
But their victory was short-lived. The ruling majority, once so complacent, now fought back with a ferocity born of desperation. The community was plunged into civil war, with the Gates and the former rulers battling for control.
In the end, neither side emerged victorious. The community was left in ruins, its once-thriving population decimated by the conflict. The survivors were left to pick up the pieces, struggling to rebuild their society in the aftermath of the war.
But even in the face of such devastation, the Gates continued to push for change. They refused to return to the old ways, determined to create a new society based on individual freedom and autonomy. And though their path was fraught with challenges, they remained steadfast in their belief that one day, they would succeed in creating a world where every person was in charge of their own destiny.
---
"Gates" had been used in an earlier prompt and story. This happened at least three or four times.
I read The Spirit Level a few years back. Some notes:
a) The writers point out that even though western countries have had a dramatic rise in economic productivity, technological development, and wages, there haven't been a corresponding rise in happiness among westerners. People are richer, not happier.
b) They hypothesize that economic growth was important up to a certain point (maybe around the 1940s for the US, I'm writing from memory here), but after that it doesn't actually help people. Rising standards of living can not help people live better.
c) And!, the writers also say that economic growth has actually led to an increase in depressions and other social ills, in rich countries.
d) Their main argument is however that equality/inequality is one of the most important factors that determines how happy people are in rich countries - and that it strongly influences the outcome of various social ills (such as the prevalence of violence, mental illness, and teenage pregnancy). Rising inequality has resulted in a broken society.
e) The core of the book are some cross-sectional studies of (i) some rich countries that fit certain criteria and (ii) the fifty states of the US, where they compare how well some social measurement (e.g. thefts per capita) correlate with the average wage and some inequality measure.
f) The writers do not present any numbers on how these variables correlate.
g) Instead, the writers produce a graph, for say "mental illness per capita", with one axis saying how prevalent the problem is ("many" vs "few") and the other axis measuring either the wage-level or the inequality level ("high" or "low"). And they also produce a line that is supposed to measure the strength of the correlation. (I didn't note at the time what exactly kind of regression analysis they did, but, again, they didn't produce any numbers).
h) Usually, they say that variable X wasn't correlated with the wage-level - but that it was correlated with the inequality-level.
i) Except for "health", they found a positive correlation between it and the wage-level.
j) Even though they found a correlation between social variable X and inequality, sometimes the most unequal society performed better than the most equal society (of the countries in the sample).
Some criticism of the book:
1) They state, but don't show that economic growth won't help people in the future - even if you accept their belief that it has had negligible or negative effects on people's happiness today.
2) The cross-sectional analysis has at least two problems. The first is that they don't tell you how correlated inequality is with some social ill. Maybe a 1% increase in inequality would increase the rate of teenage births by 2%, 20%, or 200%. Who knows?
(Furthermore, some writers say that they can not find these correlations, that they disappear if you include more countries, and that some social variables seems to be cherry picked (expenditure on Foreign Aid is used as a proxy for a virtuous society, but private expenditure to poor countries is not used). I haven't checked the validity of these claims, however.)
The second is that the writers don't show that the correlation (if it exists) really shows that higher inequality brings about the social ills they discuss. A relatively simple test they could have done would have been to see if a particular problem was correlated with inequality in a society through decades or centuries. That is, can inequality explain the rise and fall of, for instance, the homicide rate within a particular country? If you look at inequality as how much the 10% owns of GDP....then the historical record shows that it doesn't move in tandem with the homicide rate, for instance, for England & Wales, Sweden, and France. Inequality doesn't seem to influence the homicide rate at any visible level. And maybe some more thoughtful analysis will show its influence. ... Or it could be dwarfed by other factors. Or it has different effects depending upon what ideologies people have adopted.