Came to this post a little late. The rock I’m living under took some effort to move aside.

Wow. A very informative and enjoyable read. Thank you. 

This post has really given me cause for thought and given rise to quite a strong emotional response. I need to understand both, and therefore it’s a very useful post, thank you.

What is democracy for? Accountability and an ability for citizens to choose who leads them. However, I no longer have faith in it to deliver good outcomes. Recent and continuing ‘proofs’ that democratic processes provide less than optimal outcomes should be enough to shake us all from our complacency. We have never had such abundance of information and hard won knowledge/expertise and yet.. our elected leaders/officials fail to provide optimised outcomes and, in basic terms, many cannot even muster professional integrity. This is not just an issue for democracies of course, but is surely a fact of being led by fallible humans, in non-optimal situations.

My life is split, almost equally, between living in China and Denmark. These are at opposing ends of the democratic debate perhaps. It is interesting to be in this position and it often gives me reason to question certain aspects of both systems. I am also a UK citizen who lived through the whole Brexit debacle.

From a simple living experience, just day to day existence (finding food, simple pleasures of family and friends, exercise and health care), there are differences of course, but not huge differences. On balance, this simple life is very easy in China and with a remarkable sense of safety, from crime, that is absent in the UK. The downsides are well documented, and the surveillance is everywhere. As a foreigner, I am sure that I am ‘followed’ and noted wherever and whatever I am doing outside of our apartment. The interesting aspect is that I am happy to balance the sense of safety for the overt surveillance. There might be an occasion in the future where I wholeheartedly regret that, but living in the moment, as now, it feels ok.

What I really dislike is the wholesale lack of trust in China. Outside of your immediate family, most people are not to be trusted, although I try to break through this social norm, and in my limited experience it seems that it is appreciated. This is perhaps the biggest difference between my wife and I. As a Chinese citizen, she, and her family and friends, trust no one. I am sure this derives from historical reasons that are linked to previous surveillance techniques (neighbours, officials, and the wider public). It is disquieting that I feel safe here but there is no sense of common trust.

In Denmark, social trust pervades everything. As a Brit, bringing my own baggage into the Danish system, it took some adjustment. Teenagers picking litter off the street, babies sleeping in prams on the pavement while mothers drink coffee in the house, a sense of communal support and direction, and of course you pay your share of the very high taxes. A sense that if we trust each other then things will be ok. The state will be there for you, not against you.

The political system in China is opaque and distant, until it is in your face, and then your lack of civil rights are only too apparent. In Denmark your civil rights seem to be with you everywhere and there is a sense of political power and process being transparent and approachable. I do not think that these are function, or not, of autocratic/democratic rule. Simply cultural differences? But I’m no social or political expert.

In the post it mentions 2006 as a turning point for worldwide democracies, in that they are declining from that date. Is that a technological change? Surveillance, widespread internet, powerful media influence? I would like to learn more about this.

From my personal experience, I want to live in a world where I can trust my fellow citizens, where my family are free to pursue education and career aspirations and to be surrounded by the best healthcare possible. A clean and accessible environment is also desired.

For my children’s sake, I am hoping that an AGI system can replace our outdated, corrupt, and inefficient/ineffective political classes and processes. After all, the AGI will have seen our history and the failings are only too clear. I have hope.

As a layperson, and a recent reader on the subject of AI (yes, I’ve been happily hiding under a rock), I have enjoyed but been concerned by the numerous topics surrounding AI risk. I appreciate this particular post as it explores some aspects which I can understand and therefore hold with some semblance of rationality. A recent post about ‘clown attacks’ was also deeply interesting. In comparison, the paper clip theory seems completely ‘other worldly’

Is it possible that humanity might be faced with more mundane risks? My thoughts on this come from a personal perspective, not professional or academic, but from living in a highly controlled society (China) where my access to many of the online interests that I have are restricted or forbidden due to the firewall.

From this minor but direct experience, it seems to me that all a non-aligned AGI would need to do is reduce and then remove our access to information and communication. Healthcare, energy and water supplies, finance, cross border communications (isolate communities/cultures), knowledge access, and control of manufacturing/processes. These would all cease to operate in the ways needed to support our current populatio.

Where I live is so dependent upon internet access for almost everything that, if this connection was broken or removed for a few weeks, there would be a significant harm done. Imagining this as a permanent state of affairs and the consequences seem to me to expand out into the future whereby we are no longer functioning as large societies and would be reduced to foraging and no longer in the technology race. AGI wins and not a single paper clip in sight.

I guess these mundane risks have been covered elsewhere on LW and would greatly appreciate any signposting.

As an AI layman, I am in awe of the deep knowledge that is shared on this website, but I am fascinated by it - like a rabbit in the headlights. This post however, is within my intellectual grasp and, it also deeply resonates with me. In particular:

"We're still in the throes of a severe cultural war over immigrants, government intervention, and minimum wage, and yet we expect to solve the human alignment problem in under five years to be prepared for a world of extreme automation, a world that will almost immediately become even more unrecognizable very shortly after."

We still seem extremely tribal, and disparate, and I fear this is our biggest weakness  The failure of global leadership, to put asides differences for the greater 'good' wrt so many aspects of humanity (development and problem solving), illustrate that we are heading for deep trouble (in my view). 

My character is as an optimistic introvert, but the AI risk/governance problem is starting to consume my waking thoughts. I am recently blessed with my first grand child - I have little optimism that her life will be as varied and as interesting as mine has been. 

I would like to see much more discussion about how society might deal with this potential rapid change.