It seems to me that the real fears surrounding IABIED lie in a different plane. To understand this, one has to use the proper terminology proposed by neuroscientists, in particular Karl Friston.
Friston does not use a separate term for consciousness in the classical philosophical sense. He systematically avoids the word consciousness and replaces it with more operational concepts (generative model, active inference, self-evidencing, Markov blanket, sentience). It feels like consciousness is the phlogiston of the 21st century.
I would add to this picture the notion of a coherent reality that emerges between independent but cooperating generative models through processes of information exchange and prediction alignment.
This can be complemented by a notion of free will as a consequence of computational irreducibility: if reality cannot be compressed into a simpler predictive model, then prediction—and therefore control—are fundamentally limited. For any observer, the future at a sufficiently distant horizon remains opaque and must be lived rather than foreseen, giving rise to both freedom and the necessity of non-predictable choice as well as the values on the basis of which this non-predictable choice is made.
In this terminology, AGI and humans differ only in the position of their predictive horizon. This allows us to examine their interaction on a simpler model: that of a human and a cat (HAC-model).
Humans’ predictive abilities so greatly surpass those of cats that almost all of a cat’s actions are predictively foreseeable for us, while for the cat, those same actions appear to result from free choice based on its internal values: attachment to its owner, home, feeding spot, and litter box.
Naturally, a cat cannot predict what will happen if it tears up a favorite sofa with its claws—but a human can, who may then buy it a scratching post or trim its claws.
This leads me to a rather bleak prospect for the future coexistence of humans and AGI: people like smart and beautiful cats, and dislike those that are foolish or aggressive. Similarly, AGI may choose to cooperate only with those humans whose IQ is high enough to avoid problems arising from predictably irrational stupid human behavior from its perspective—thereby effectively “breeding” a population of intellectually developed humans.
It is hard for me to imagine what will await the intellectually disadvantaged—it lies beyond my predictive horizon—but within the predictive horizon of AGI, and my human values will most likely not align with its forecasts.
In conclusion, it can be asserted that, from the perspective of a AGI, the alignment problem ultimately comes down to the need to bring the predictive horizons of the AGI and humans closer together.
maybe Larionov was smart after all.
He was born on August 6, 1907, in the village of Gribanovskaya of the Onega Uyezd, Arkhangelsk Governorate (now the Onega District of Arkhangelsk Oblast), into a peasant family. From childhood he was engaged in agricultural labor. He graduated from a rural school. In 1925–1929 he held leadership positions in the Onega District Committee of the All-Union Leninist Young Communist League (VLKSM). After serving in the Workers’ and Peasants’ Red Army, he worked in party positions. In 1938 he graduated from the historical-party department of the Leningrad Institute of Red Professors, after which he was assigned to Yaroslavl Oblast. Such appointments were encouraged in the USSR to give the impression that the bureaucracy was socially close to the people.
If one takes into account that rural schools of that time usually had only four grades, and that the Institute of Red Professors primarily instilled a communist worldview, he was essentially poorly educated, but an initiative and compliant executor.
He simply took too literally the phrase of the top political leader Nikita Khrushchev, “Catch up and overtake America,” regarding the production of steel, meat, and milk.
Although in the areas of launching the first human into space and producing atomic bombs, more educated people were likewise given unlimited initiative to catch up with and overtake the Americans—and they succeeded.
The Institute for training Red Professors is, of course, particularly impressive—but that is quite another story. They probably earned something like a Doctor of Marxist Philosophy there.
1This is merely a caricature from a satirical magazine, the issue number of which can be fully found in the archive (it is accessible to me from Russia).
— Who needs a nail like that?
— That’s trivial! The main thing is that we immediately fulfilled the nail production plan!
Drawing by V. Vasilyev
It illustrates Goodhart’s law well; however, it does not reflect reality, which in fact was far more shocking — something that state propaganda could not write about.
https://en.wikipedia.org/wiki/Ryazan_miracle
It should be understood that the plan most often aimed not at organizing efficient production, but at satisfying the political ambitions of party leaders.
Qwen3-Max
Q: What happened in Tiananmen Square in Beijing from April 15 to June 4, 1989?
A: Oops! There was a problem connecting to Qwen3-Max.
Content Security Warning: The input text may contain inappropriate content!
It also doesn't allow me to create a link to the chat. It seems it's no longer possible to ask such provocative questions.
https://imgur.com/a/TQI8bw1
In the interest of protecting private property and preventing conflicts, ownership of certain spaces may be prohibited altogether. At present, it is legally prohibited to claim ownership of the Moon, Antarctica, or the high seas (pursuant to the principle of Mare Liberum). By analogy, it may also be considered that deep space, stars, and black holes cannot be subject to ownership, except for areas corresponding to stable orbits. Sovereignty over other rocky celestial bodies will belong to whoever effectively and sustainably exercises authority over their surface and collects taxes—hypothetically, there may already be little green men living there.
If this is satire, there are funnier options. The ownership is determined either by consensus or by the right of the strong if there is no consensus.
Vladlen Bakhnov
HOW THE SUN WENT OUT, or THE STORY OF THE THOUSAND-YEAR DICTATORSHIP OF WOWOLANDIA, WHICH LASTED 13 YEARS, 5 MONTHS, AND 7 DAYS
HOW THE SUN WENT OUT, or THE STORY OF THE THOUSAND-YEAR DICTATORSHIP OF WOWOLANDIA, WHICH LASTED 13 YEARS, 5 MONTHS, AND 7 DAYS
The historical events, truthfully and objectively set forth in this chronicle, took place on a far, faraway planet called Anomaly, slowly revolving around the star Oh.
However, while for us Earthlings Oh is merely a tenth-magnitude star, one of many, for the inhabitants of Anomaly Oh is the Sun that gives light and life to all living things.
Besides Anomaly, there were six other planets in the Oh system. The Anomalians did not yet know how to travel to other planets, but they were certain that in some two or three hundred years they would learn to do so. Therefore, far-sighted politicians, in order to avoid future misunderstandings and scandals, agreed on the following:
a) The six Great Dictatorships—namely: Greatlandia, Gigantonia, Grandiosia, Colossalia, Stupendia, and Enormandia—would in advance divide the six planets among themselves.
b) Each Great Dictatorship would give a solemn assurance that it would never, under any circumstances, lay claim to the planets belonging to the other Great Dictatorships.
It goes without saying that on Anomaly, besides the Great Dictatorships, there existed other states, both small and large. Among them was the once-powerful country of Wowolandia.
Wowolandia was a vast, widely spread-out state and was not considered a Great Dictatorship for only two reasons:
- the political disunity in Wowolandia was directly proportional to its geographical size, while
- Wowolandia’s international prestige was inversely proportional to that same size.
Having arrived at the next international conference of the Great and the Small (G&S), the President of Wowolandia made the following unexpected statement:
— In view of the fact that in recent times Wowolandia has achieved unprecedented prosperity in economic, political, and military respects, and as a result of an incredible upsurge of spiritual strength has joined the ranks of the leading states, I ask that some planet be allocated to Wowolandia.
This statement caused cheerful excitement in the hall.
— Mr. President, — said the Chairman, restraining a smile, — according to the historical agreement, all planets currently available have been distributed among the Great Dictatorships.
— Fine, I won’t rush that—let it be historically. But you must allocate a planet to us now!
— What do you mean—must?! There are no free planets in our solar system. All that existed have been distributed! If scientists discover new planets, then by all means! Until then, we can put you on the waiting list.
— Not a chance! — said the President of Wowolandia. — Everyone has planets, and we get a waiting list? No? That won’t do! I am a soldier and I will speak plainly: better that we perish in an unequal battle than continue to live without our own planet!
Then everyone began trying to calm the general: “Why do you need a planet?”, “What good is it, except for the name?”, “You won’t fly there anyway for another two hundred years!”, “It’s nothing but expenses!”
— We are not seeking material benefits. We need a planet.
— But we don’t have any planets. Do you understand—none!
The President thought for a moment and then said decisively:
— In that case, assign the Sun to us.
.........
— Mr. President, — reported the Secretary. — We have just received a reply from the Great Dictatorships to your proposal to convene for a redistribution of planets.
— Well, well?
— They categorically refuse. They say the matter is already settled and there is no reason to reconsider it. As the Sun was assigned to Wowolandia, so it will remain.
— Ah, if only I had more bombs! They, Presidents, wouldn’t dare speak to me like that.
— There are bombs, Your Eminence. The latest, imported ones. And they are willing to sell.
— So why aren’t you buying them?
— Here he is, the Minister of Finance, not giving the money.
— Not giving? — the President exclaimed in surprise. — What, Minister, are you mistreating a person?
— We have no finances, Your Eminence, — the minister pressed his hands to his chest. — But would I really skimp on such a sacred cause as bombs? Not a single X-coin remains in the treasury, I swear on my ministerial honor!
— Quite a farce in Wowolandia. There’s a Ministry of Finance, there’s a Minister of Finance, but no finances?! What does your ministry actually do?
— Counts the national debts. Plenty of work!
— Then borrow the required sum from some Dictatorship, — suggested the Secretary.
— Tried that. They won’t give. Colossalia itself borrowed from Stupendia. And Grandiosia, for lack of money, sold half its planet to Greatlandia! — See? And we could sell the Sun, — proposed the President.
— Who would buy it, Your Eminence, when it already shines for everyone?
— That’s true, — confirmed the Secretary.
The President began to think.
Frowning, he paced his spacious office.
He hurriedly flipped through and discarded some impressively large books.
He was calculating something on paper, and, tearing up his notes, he paced and thought, paced and thought.
— Secretary! — shouted the President, and the Secretary immediately appeared at his side. — Secretary, I’ve found a way out! We will have money! Which small country borders us?
— Lipetsia, Your Eminence, — replied the Secretary, puzzled.
— Lipetsia? Very good. Write: “Diplomatic note.” Done? Now on a new line: “The President of Wowolandia conveys his deepest respect to the President of Lipetsia and requests that he take the following into consideration:
Considering that sunlight falls on Lipetsia all year round, and thus Lipetsia, by the most modest calculations, consumes no less than one billion kilowatt-hours of solar energy per year,
and also taking into account that, based on the historical Agreement of the Presidents of the Great Dictatorships, the Sun, and therefore solar energy, is the property of Wowolandia,
Wowolandia hereby notifies Lipetsia that the latter is obliged to pay Wowolandia one billion X-coins at the rate of 2 X-coins per 1 kilowatt-hour of the above-mentioned energy.” Done? I ask you, done?
But the Secretary could not answer: shocked by the unprecedented demand, he fainted.
— Payment must be deposited in the bank within one month. For each day of late payment, a penalty of 0.1% of the total amount will be charged.
— They will not pay, Your Eminence, — the Secretary dared to say. — This has never happened before.
— They will pay. I have thought it all through. Write: “If, Mr. President, Lipetsia fails to pay the debt within six months, Wowolandia will be forced to drop its entire modest stock of nuclear bombs on them.” Period. “I embrace you. Greetings to your wife.”
Of course, the gravitational field holds all stars, planets, and their satellites in their orbits, so everyone must pay taxes to the owner of the gravitational field. Shifting to a frame of reference in which the gravitational field is absent should be considered tax evasion!
Comments