The use of inconsistency you described requires someone to already have power. Otherwise, why would people bother tracking their current opinion?
The article feels like an excuse to discuss politics, under pretext of discussing AI safety. I get it, you don't like Trump. But how much of what he does is unique, and how much is a case of "our politicians are charismatic leaders, their politicians are manipulative demagogues"? For example, does the pattern "publicly joining our side is the way to stop attacks" not happen in woke politics? Or that yesterday's dogma is today's heresy? (That is, even if Trump is a rare type of a demagogue, still a subset of what he does could be "business as usual".)
I am using Trump merely as an illustrative example of techniques.
My more immediate concern is actually the ability of China to shape US opinion through TikTok.
I've aimed to have this read equally well whether or not you like him.
hover to invoke Crocker's Rules:
you failed miserably
This post seems to focus too much on the denotative content of propaganda rather than the social context in which it occurs. Effective propaganda requires saturation that creates common knowledge, or at least higher-than-first-order knowledge. People want to believe what their friends believe. If you used AI to generate political messages that were custom-tailored to their recipients, they would fail as propaganda, since the recipients wouldn't know that all their friends were receiving the same message. Message saturation and conformity-rewarding environments are necessary for propaganda to succeed; denotative content barely matters. This makes LLMs practically useless for propagandists, since they don't establish higher-order knowledge and don't contribute to creating an environment in which conformity is socially necessary.
(Overemphasis on the denotative meaning of communications in a manner that ignores their social context is a common bias on LessWrong more generally. Discussions of persuasion, especially AI-driven persuasion, are where it tends to lead to the biggest mistakes in world-modeling.)
Social media has proven more than capable of creating effective social contexts for persuading people.
LLMs are perfectly capable of operating in these social contexts. Particularly if they have (as in the case of TikTok and China) the support of the owner of the site.
Do you have specific cause to believe that LLMs will fail to persuade in these social contexts?
Agree with a lot, but major concerns:
And the bigger the shift, the more it reinforces for the followers that their reality is whatever the leader says.
Eventually people's desire to comply overcomes their sense of reality. [...]
Trump also changes positions regularly.
[...]
That explains why Ted Cruz became Trump's ally. It was not despite Trump's humiliation of Cruz, but because of it. The Corruption of Lindsey Graham is an excellent in depth read on how another opponent became a devoted supporter. [...]
Do you really believe Cruz' & co's official positions have anything to do with their genuine beliefs? That seems a weird idea. Would you have thought the same thing about Tucker Carlson before his behind-the-scenes hate speech about Trump got disclosed? I think the reality of big politics business is too obvious to claim this is in any way about true beliefs.
2. Another point of skepticism is on supposed "70 million devoted followers who hang on [Trump's] every word".
I have the impression (maybe simply a hope), a (large) bunch of these may not be as fully brainwashed as we make it to be. I can too easily imagine (and to some degree empathize with) persons not buying all that much from Trump at all, but still liking simply his style, and hating so much the conventional politics, with their anyway constant lies but more hidden than with Trump, that as protest they really 'love' him a bit compared to the (in their view) not-less-dirty but more uncanny-as-carefully-hidden.
3. Finally, on AI specifically:
Of course, this leaves open the scope for big AI guys pushing AI safety regulation in a direction that specifically serves them instead of (only) making the world saver. That would barely surprise anyone. But it substantiates "AI safety as a PR effort" about as much as the fact that 'climate scientists would loose their job if there was no climate change' proofs that climate change is a hoax.
You make a good point that Cruz et al may have different beliefs than they portray publicly. But if so, then Cruz must have had a good acting coach in late 2018.
About 70 million followers, you're right to call me out for overstating it. But according to polling, that's how many people believe that the 2020 election was stolen at the ballot box. So far he has lost dozens of election challenge cases, key members of his inner circle have admitted in court that there was no evidence, and he's facing multiple sets of felony charges in multiple jurisdictions.
I think it is reasonable to call someone a devoted follower if they continue to accept his version in the face of such evidence.
On AI safety, we can mean two different things.
My concern is with the things that are likely to actually happen. Hence my focus on what is supported by tech companies, and what politicians are likely to listen to. That part I'm sure is mostly regulatory capture.
I did acknowledge, though not loudly enough, that there are people working in AI safety who truly believe in what they are doing. But to the extent that they don't align with the vested interests, what they do will not matter. To the extent that they do align, their motivations don't matter as much as the motivations of the vested interests. And in the meantime, I wish that they would investigate questions that I consider important.
For example, how easily can an LLM hypnotize people? Given the ability to put up images and play videos created by AI. Can it hypnotize people then? Can it implant posthypnotic suggestions? In other words, how easily can an existing social network, with existing technology, be used for mass hypnosis?
Update. I forgot to mention that Andrew Ng's accomplishments in AI are quite impressive. Cofounder of Google Brain, taught machine learning to Sam Altman, and so on. I might wind up disagreeing with some of his positions, but I'll generally default to trusting his thinking over mine on anything related to machine learning.
If you're willing to pay, you can read a stronger version of his thoughts in Google Brain founder says big tech is lying about AI extinction danger.
Thanks!
Cruz: I think a model where being a terribly good liar (whether coached, innate, or self-learned) is a prerequisite to for becoming a top cheese in US politics, fits observations well.
Trumpeteer numbers: I'd now remove that part from my comment. You're right. Shallowly my claim could seem substantiated by things like (Atlantic) "For many of Trump’s voters, the belief that the election was stolen is not a fully formed thought. It’s more of an attitude, or a tribal pose.", but even there upon closer reading, it comes out: In some form, they do (or at least did) seem to believe it. Pardon my shallow remark before checking facts more carefully.
AI safety: I guess what could make the post easier to understand, then, is if you make it clearer (i) whether you believe AI safety is in reality no real major issue (vs. only overemphasized/abused of by the big to get regulatory advantages), and if you do, i.e. if you dismiss most AI safety concerns, (ii) whether you do that mainly for today's AI or also for what we expect for the future.
Ng: In no way I doubt his merits as AI pioneer! That does not guarantee he has the right assessment of future dangers w.r.t. the technology at all. Incidentally, I also found some of his dismissals very lighthearted; I remember this one. On your link Google Brain founder says big tech is lying about AI extinction danger: That article quotes Ng on it being a "bad idea that AI could make us go extinct", but it does not provide any argument supporting it. Again, I do not contest AI leaders are overemphasizing their concerns, and that they abuse of them for regulatory capture. Incentives are obviously huge. Ng might even be right with his "with the direction regulation is headed in a lot of countries, I think we’d be better off with no regulation than what we’re getting", and that's a hugely relevant question. I just don't think it's a reason to dismiss AI safety concerns more generally (and imho your otherwise valid post loses power by pushing in that direction e.g. with the Ng example).
I believe that AI safety is a real issue. There are both near term and long term issues.
I believe that the version of AI safety that will get traction is regulatory capture.
I believe that the AI safety community is too focused on what fascinating technology can do, and not enough on the human part of the equation.
On Andrew Ng, his point is that he doesn't see how exactly AI is realistically going to kill all of us. Without a concrete argument that is worth responding to, what can he really say? I disagree with him on this, I do think that there are realistic scenarios to worry about. But I do agree with him on what is happening politically with AI safety.
Sam Altman recently said:
i expect ai to be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to some very strange outcomes
I believe that he is absolutely right. Superhuman persuasion can be achieved by having LLMs consistently apply existing capabilities to known persuasion techniques. Some of these techniques are already proven to work at scale. Others have demonstrated effectiveness, and can more easily be applied at scale by AI. This makes superhuman persuasion a straightforward technology proposition.
I will look at this from the point of view of how AI can enable someone possessing the positions, skills, and desires necessary to attempt to create an authoritarian regime. In other words, I am describing something that is a near-term possibility without major advances in AI. Major advances seem likely, and would only make this picture worse.
I will also focus on techniques whose effectiveness has been proven by repeated human success. I will focus most of all on dictators and demagogues. Because he is so familiar, I will cite Donald Trump as an expert practitioner. This is not support for, or criticism of, him. I've aimed to have this read equally well whether or not you like him.
The information that I'm presenting is not new. Though it has come together for me as part of thinking about my blog. And I decided to post this as a response to, We are already in a persuasion-transformed world and must take precautions.
Dictators want a mass audience who is emotionally aligned with them. These people should want the dictator to be right. And ideally should find it painful to question the dictator. The result is followers who can't be convinced by rational argument.
This requires conditioning System 1. So you want to speak in a way that System 1 responds to, but doesn't activate System 2. Use your speeches to deliver a key emotional message over and over again.
The necessary verbal pattern involves lots of simple and repetitive language. Avoid questions. Questions activate System 2, and you don't want that. If you can, rearrange sentences so that the most impactful word comes last. Do not worry about logical consistency. If System 2 doesn't activate, logical gaps won't be noticed.
You can personally practice this speech pattern by studying dirty talking. If you're in a relationship, you can even practice with a consenting target. You'll find that it really works.
Reading through that article on dirty talking, the dialog is simple, repetitive, and blunt. If you're emotionally aligned with the speaker, it is extremely pleasant to hear. If you're not aligned with the speaker, it feels like attempted mental rape.
Now go review some of Donald Trump's speeches. You'll find the same pattern. Though with generally less sexual content. His speeches feel wonderful if you're aligned with him. You listen, and truly believe that he will make America great again. If you don't like him, they are torture to sit through. The resulting liberal outrage is music to the ears of conservatives who want to own the libs.
Readability metrics score Trump's speeches at about a grade 5 complexity. He has by far the easiest to understand speeches of any major politician. He's also the only politician with 70 million devoted followers who hang on his every word. This is absolutely not coincidence.
You'll find the same thing in the propaganda used by successful dictators from Benito Mussolini onwards. For example, try watching Russia Media Monitor. You'll find the same verbal patterns, and the same results. The condition is literally strong enough that people would prefer to think their children Nazis than question the propaganda.
ChatGPT has no problem imitating a style like this. You just have to list the emotional buttons to hit, and tell it to copy the style of already effective demagogues.
If you want to dive deeper into this topic, Comply with Me: Trump's Hypnosis Toolkit Exposed may be worth reading.
Dictators are famously inconsistent. Most don't realize that this inconsistency helps them succeed.
The name of the game is attention. I grew up in a dysfunctional family, and the dynamic was the same. My mother always had a declared enemy. Every so often she declared a new one. Often the previous enemy was now our ally. Questioning her shifts made you suspect, and a potential target. Being aligned with the order of the day was the best way to be in her good graces. And everyone wanted to be in her good graces.
The only way to consistently get it right, was to pay close attention to her. And follow her every mercurial shift.
Dictators do the same thing. Paying attention requires their followers to constantly consume targeted media content. Creating the perfect audience for System 1 targeted propaganda. That reinforces the followers' desire to agree with the leader. Which makes each sudden shift even more effective as a demand for attention. And the bigger the shift, the more it reinforces for the followers that their reality is whatever the leader says.
Eventually people's desire to comply overcomes their sense of reality. Leading to the Orwellian reality that people really do believe the regime when it says, "We have always been at war with Oceana."
Trump also changes positions regularly. His opponents keep hoping that Trump's latest change in position will wake his followers up. That's wishful thinking. Mercurial behavior reinforces control. There is every reason to believe that it will continue working for Trump.
Someone using AI for propaganda can obviously simply manually decide when the AI should change positions. So AI does not need to figure this out for effective propaganda.
That said, I expect that ChatGPT could do an excellent job of deciding when to change positions. The ability to issue personalized appeals should let AI suck people into many small groups, and then manipulate those groups into a mass movement.
This was one of Donald Trump's key techniques for taking control of the Republican Party. Over and over again he humiliated opponents. Then he found opportunities to get them to support him. He amplified their statements of support to commit them to it. And then treated them generously.
This left his opponents with a problem. Why would they support someone who had treated them badly? Trump offered only one easy out. Accept that Trump truly is a great man. You deserved punishment before when you denied his greatness. Now that you've seen the light, you deserve When you deny his greatness, you deserve punishment. Now that you've admitted the truth, you deserve honor. And if you take this out, you've just become his loyal supporter.
That explains why Ted Cruz became Trump's ally. It was not despite Trump's humiliation of Cruz, but because of it. The Corruption of Lindsey Graham is an excellent in depth read on how another opponent became a devoted supporter. In the sectionTrump's Best Friend you see over and over again how Trump got Lindsey to compromise himself. The Kavanaugh section of Power Shift to Trump shows when he completely and utterly became Trump's man.
Please note. This happened to intelligent elites who thought that they had a clear understanding of how they could be manipulated. Flipping them generally took Trump 2-3 years. And they've stayed loyal since. Effective persuasion is not magic. It takes time. But conversely, we tend to underestimate we change over such periods of time. And therefore we overestimate how resistant we are to such persuasion.
An authoritarian leader can only get to so many people individually. Doing this at scale is harder. Groups have come up with many approaches. You can require public declarations from party members. Openly name and reward people who turn in friends and family. A cult like Scientology can give everyone individualized therapy with therapists trained to create cognitive dissonance. All of these use those who already converted to help convert more.
It seems unlikely that ChatGPT will match the best humans in the near term. I expect it to be better than most people. But a lot of people want to take control of major organizations such as a political party or country. Success requires many things to go your way, but is extremely unlucky unless you have exceptional skills. So a would-be authoritarian assisted by AI should personally develop this skill.
But what the AI lacks in individual ability, it makes up for in scale. Demagogues struggle with creating opportunities for cognitive dissonance in a mass audience. But AIs allow people to be individually monitored and manipulated. And if this technology is deployed on a mass level, this capability is an obvious target for improvement.
We want to be part of groups. Then once we feel that we are, we naturally change our ideas to fit with the group. This is the source of groupthink.
These dynamics are easiest to see in cults. For example read Brainwashing and the Moonies for how Moonies suck potential members in. Authoritarian regimes also use similar techniques. See Thought Reform and the Psychology of Totalism for a detailed description.
All of these things work far better on people who are young and impressionable. My wife still talks about how effective the Soviet Pioneer organization was on her. Ukrainians today are struggling with the mass abduction and brainwashing of their children by Russia.
For Putin it wasn't a crazy idea to capture kids, brainwash them, then send them to fight against their parents. Totalitarian regimes have a lot of experience with brainwashing. He knew that it would reliably work, about how long it should take, and how to set up the program.
How can a would-be authoritarian use AI to help?
The biggest thing that I see is that social media is full of lonely people seeking community. Social media is very good at helping them find it. But, especially when controlled by AI, social media can choose which community for them to find. It can choose which messages get boosted, which get suppressed. Participants in the community may themselves be artificially created. Through those, AI can directly make suggestions for community building exercises to reinforce the dynamics. And, over time, AI can ensure that the other techniques are used to bind people to the group, then move the group to thinking what the AI wants.
From a would-be dictator's point of view, current AI has some unfortunate limitations.
The biggest one is consistency. For example it is easy to create a chatbot to be your girlfriend. Unfortunately some fraction of conversations between a couple will lead to a breakup. Therefore there is a chance that your AI girlfriend will break up with you. Properly indoctrinating people takes more consistency that AI has.
The second one is memory. Using the same AI girlfriend example, people notice that, "Replika has the memory of a goldfish."
I don't believe that either of these needs a fundamentally better LLM. Memory can be achieved by having the LLM summarize and save information, then at appropriate times re-read it back into memory and carry on. A variety of schemes are possible for this, and this is an area of active research. This problem is probably solvable for our potential AI-enabled future overlord.
Consistency can be addressed similarly. AI may not stay on message all of the time. But it can also be prompted behind the scenes to reload the prompt. Between having good prompts, and periodic reminders of what they are, we should achieve sufficient consistency to make the technology work.
Therefore I believe that both improvements can be achieved with better tooling around LLMs, without having fundamental improvements in LLM technology itself.
You may wonder why I am not worried about known AI weaknesses like prompt injection. The answer is simple. Authoritarian techniques just need to be good enough to work for large numbers of people. As long as they are accepted at an emotional level, logical flaws will dissuade people. Therefore being able to show people what is going on won't change their minds.
For example, consider what happened when fact checkers tried making rational arguments about how often Trump bends the truth. Trump supporters did not stop supporting Trump. Instead they came to distrust the fact checkers!
Probably?
The obvious social network to pick on is TikTok. It is controlled by China. China, obviously, is an authoritarian country with a deep understanding of authoritarian techniques. And they have direct access to the most eyeballs in the young adult audience - which historically has been the most desirable target group for indoctrination.
In the last month we've had an entirely unexpected outburst of actual antisemitism worldwide. Jews are being libeled, murdered and assaulted all over the world, whether or not they personally support Israel. This is absolutely shocking for those of us who thought that antisemitism was dead and gone, a problem for other people far away from is in geography and/or time. And the age range where this is happening most obviously is the prime target for TikTok.
Was this outburst entirely a spontaneous social media phenomena, sparked by world events? Or has it been shaped behind the scenes with the help of AI? I have no real evidence either way, but I have my suspicions. I would be shocked if China wasn't trying to figure out how to create effects like this. And this is exactly what I'd expect it to look like if they did. But it could have happened naturally. So they at least have plausible deniability.
My opinion is entirely personal. I know very intelligent people who think oppositely from me. And so I encourage readers to make up their own mind.
I see AI safety as a PR effort by established tech companies aimed at getting government to create a regulatory moat for them. I'm sure that many very sincere people are part of the AI safety community. But they will only get funding to the extent that tech likes what they say, and will only get listened to to the extent that politicians hear something that they want to hear.
I admit to having a prior bias for this opinion. I've long been concerned about the ways in which regulatory capture allows established companies to take control of regulatory regimes for their own benefit. This inclines me to be suspicious when CEOs of companies who would most benefit from regulatory capture are proposing a new regulatory regime.
However I'm not the only one with some level of cynicism. As an example, well known machine learning researcher Andrew Ng has doubts about AI safety. And Eliezer Yudkowsky recently said in a piece of fiction:
...it's just a historical accident that 'AI safety' is the name of the subfield of computer science that concerns itself with protecting the brands of large software companies from unions advocating that AIs should be paid minimum wage.
So as you think about it, assign a prior to how likely it is that this is regulatory capture, versus a genuine concern. If you want to do it right, you should assign that prior per individual because it is almost certain that different participants have different motivations. Then make your own evaluations of which theory each action that they take more strongly supports one theory versus the other.
My prior starts off biased towards regulatory capture because I know how well-known the dynamic is in tech. And I see the financial incentives to think that way. My opinion has only been strengthened by proposed regulations. They seem to me to do more to make entering the market expensive than they do to solve real problems. This strengthens my opinion.
However my opinion would shift if I saw more concern about what I'm discussing here. That is, thinking about how existing technology could take advantage of known flaws in humans. Rather than discussion of what shiny future technologies might create problems.
Your opinion may differ from mine for a variety of reasons. For example it seems obvious to me that people should be thinking along the lines that I outlined above. What could be more natural than that people should think the way that I do. But you may conclude that I'm just weird in thinking the way that I do.
But, whatever you think of my potentially weird thinking, I hope that others at least find this an interesting thing to think about.