I've argued for "hallucination" -> "Runge Spikes"; see https://news.ycombinator.com/item?id=43612517

"Runge Spikes" is still a metaphor, rather than a technically accurate description. Its strength is that it gets away from using a metaphor from human illness. Metaphors from human illness smuggle in an assumption that LLMs are faulty humans, rather than technology doing its thing.

Eliezer's essay "Disputing Definitions" is didactic writing, but one can also read it as a lament. He even uses the word "mournful". He ends his essay, like I started my comment, with making up two new words, intending to head off what he calls the "Standard Dispute". His version is tongue-in-cheek. His words are "alberzle" and "bargulum" and there is a time machine.

His essay is excellent, but how does an essay from 2008 need updating for the LLM era? He is lamenting both what is in the training data and what is missing from it.  People dispute definitions. They fail to invent new words to head off these disputes.

My claim is that many of our "Standard Disputes" have their origins in linguistic poverty. Enrich language with new words, targeting the ambiguities that we quarrel over, and the problems are solved at source. But turn to an LLM for help and it will help you to write fashionable prose. Since neologism have never been in fashion, (and are subject to mockery, see https://xkcd.com/483/) the LLM will not suggest any. Rather, it will guide you down the path of the "Standard Dispute", leading you away from low hanging fruit.

For a whimsical speculation, imagine that the New York Times publishes a list of one hundred new words to enrich political discussion. Inventing new words becomes all the rage. In 2026 human authors who want to join in the craze will have to invent their own. In 2027 the linguistic patterns involved will be in the training data. In 2028 egglogs (egregious neologisms) are the hallmark of AI slop. In 2029 neologisms are banned and by 2030 we are back to disputing definitions, just like we did in 2025.

I agree with the main post. My narrow point on neologisms all that I have to add.

People often let language do their thinking for them. For example, consider the phrase "this has happened before". A discussion of artificial intelligence leading to technological unemployment will devolve into "this has happened before" versus "this time is different".

But "this has happened before" muddles together two different ideas that I call *restrep* and *maytip*. *Restrep* is what happens at the start of a second game of chess. The board is cleared and the pieces set out in the standard starting position. Reset and repeat. The second game may start with exactly the same opening as the first game. One guesses that the player who lost the first game will be the first to depart from exact repetition and try something new. By contrast *maytip* is the story of pack horses being displaced by canals being displaced by railways being displaced by trucking. Since there is no reset, things are accumulating and we may reach a tipping point eventually, even if this time is *not* different.

If we had better language, we would talk of technological unemployment in terms of maytip and whether this time is really different. We would not confuse the strong evidence that the past provides in *restrep* with the weak evidence that the past provides in *maytip*.

LLMs are even more deeply trapped in language than we are. Language is all they have. This is sad because us humans could really do with some help in escaping the language traps that we fall into. Sometimes we build those traps ourselves. Think of how the abortion debate turned into pro-choice versus pro-life when word creation became a new front in the culture war. Other times we use traditional phrases such as "defensive alliance". Look at this attempt https://www.themotte.org/post/1043/splitting-defensive-alliance-into-chaining-alliance to split "defensive alliance" into "chaining alliance" and "isolating alliance". The comments spot the relevance to 2025 with the current war in Ukraine, and pursue the object level arguments. As humans we are quick to spot which political faction gains an advantage from existing language and they resist any attempt to improve language so that we can be less wrong.

That leaves us with a dream about artificial intelligence, that it would have an inner voice, speaking a language of thought unrelated to the natural languages of humans. It would find our words ambiguous, and help us to go beyond them. It is currently a broken dream. LLMs understand the world through our existing vocabulary and suffer the limitations that those words bring.
 

My take on the conflict theory analysis is that the reserve army of brutal thugs is a valuable resource for avant-garde revolutionaries. Think 1917 Russian revolution. Its was a close run thing with a brutal civil war. Typically the avant-garde don't have the numbers. They may win power, but not have the numbers to hold on to it. They need to put boots on the necks of counter-revolutionaries. Since their tests for counter-revolutionariness have too many false negatives, they have to go large and put boots on the necks of the general population. Where do they find the feet to fill the boots? They release brutal thugs from prison to provide the muscle for the NKVD, KGB, Stasi, etc.

It is a very dangerous game. The avant-garde revolutionaries need to retain control of their brutal thugs. The thugs need to be kept divided. If some get ideas above their station, others are sent to kill them. But the Russian revolution and the French revolution both ate themselves. One faction within the revolutionary avant-garde sends their tame thugs to kill a rival faction within the avant-garde. The death toll rises and Stalin or Napoleon comes out on top.

I'm unclear on the causal connections here. Perhaps opposition to the death penalty is all high minded mercy. When the revolution comes, it is an unfortunate accident that the revolutionaries are gifted a reserve army of brutal thugs to help them consolidate their power. Or perhaps there are some strategic thinkers covertly funding the merciful people naturally inclined to oppose the death penalty. The money boosts the opposition to the death penalty, enough for mercy to defeat prudence.

It is not just domestic revolutionaries that one has to worry about. When the USSR took over Eastern Europe at the end of WWII, releasing brutal thugs from prison, to provide the muscle for the secret police, was one of the techniques used to impose the new communist governments.

You are over-simplifying Bayesian reasoning. Giving partial credence to propositions doesn't work; numerical values representing partial credence must be attached to the basic conjunctions. 

For example, if the propositions are A, B, and C, the idea for coping with incomplete information that every-one has, is to come up with something like P(A)=0.2, P(B)=0.3, P(C)=0.4 This doesn't work.

One has to work with the conjunctions and come up with something like 

P(A and B and C) = 0.1

P(A and B and not C) = 0.1

P(A and not B and C) = 0.1

P(A and not B and not C) = 0.2

P(not A and B and C) = 0.1

P(not A and B and not C) = 0.1

P(not A and not B and C) = 0.1

P(not A and not B and not C) = 0.2

Perhaps I should have omitted the last one, for the same, adds up to one reason that I omitted P(not C) = 0.6. One actually has to work with seven numbers not three.

Ordinarily I would approve of simplifying Bayesian reason in this way; it helps you get to your point quickly. The reason that I criticize it as an over-simplification is that you proceed to talk about fuzzing the propositions in four ways: vagueness, approximation, context-dependence, and sense vs nonsense. Propositions or basic conjunctions?

A big problem with Bayesian reasoning is that the number of basic conjunctions increases exponentially with the number of propositions. This makes Bayesian reason rather impractical. One must resort to various ugly hacks to tame this exponential explosion. I believe that the problem is not actually with Bayesian reasoning, but with having incomplete information. Any attempt to cope with missing information will suffer from this exponential explosion and need hacky fixes.

Maybe you can cope with vagueness, approximation, etc, by fuzzing the propositions, but when you try to accommodate missing information you will have to work with basic conjunctions. If proposition A has category boundaries that are fluid and amorphous in one way, and proposition B has category boundaries that are fluid in a different way, you will need some kind of product structure on fluidity so that you can cope with "A and B" and also "A and not B", "not A and B", and finally "not A and not B". Maybe you can postulate that the fluidity of A is just the same whether B is true or false, but this is basically a hack to try to contain the exponential explosion of the inherent difficulties.

I started going barefoot in the streets of Edinburgh in February 2000. Eventually I wrote a little web page explaining myself. I didn't want to duplicate what was on the Society For Barefoot Living website, so I narrowed my focus to a single aspect. Twenty four years later, I still go barefoot nearly all the time. Rescuing the text to paste it here, I notice that it has stood the test of time very well :-)

Hard surfaces

Modern life involves much walking on hard surfaces, pavements, reinforced concrete floors, steel decking, and it is worth pondering whether shoes provide adequate cushioning. In modern shoes, your heel hits the pavement first, before the rest of your foot. With the pace of modern life hit is the right word, and the cushioning provided by the heel of your shoe as you pound the pavement is at issue.

I think that the cushioning is inadequate and cannot be improved because the basic concept is faulty. One is better off spending a few months learning to walk barefoot.

Wait a minute! There is no cushioning at all under the heel when you walk barefoot; how can that be an improvement? It is time to get technical and explain the difference between a soft material approach to cushioning and a mechanical approach.

A soft materials approach to cushioning

Softness is a three dimensional phenomenon. When you compress a material it squeezes out sideways. Typically it pushes out about a third the amount of compression. This number is called Poisson's ratio. This is the beginning of the story, not the end. Try holding a pan scourer, one of those little blocks of sponge, between the palms of your hands as though you were clapping. Squeeze and it compresses. You knew that. Now try bringing your little fingers together without moving your thumbs. It resists being squeezed, but does very little to keep your palms parallel. Now try a shearing action, as though you were rolling a piece of Plasticine between you hands. You encounter a little more resistance than you did when squeezing, you will need to squeeze a little to stop it sliding. Now try a twisting action, by pointing the fingers of one hand down and the fingers of other hand up. Again you will need to squeeze a little to stop the sponge from sliding. The softness that cushions your clapping to silence has brought with it flexibility to five other motions. A nice, soft shoe heel would wobble all over the place and be too squishy to walk on.

A mechanical approach to cushioning

To experience a mechanical approach, sit on the wing of your car. Your weight makes it sink an inch or two. Isn't that the compression of the air-filled rubber tire? Well, it is in plain view, so look and see. It has hardly squished at all. To find out where the motion has come from you have to look up inside the wheel arch so that you can see the suspension. Most of the motion has come from a mechanism. Your weight has made a lever pivot about its hinge so that it stretches a spring. There is an important technical reason for car makers taking this expensive mechanical approach, instead of relying on soft materials. The mechanism decouples the different motions. The stiffness of the motion that makes the hinge pivot is determined by the spring. The stiffness of other motions is determined by how solid the hinge is. The manufacturer can chose the softness of the spring to suit the single motion that the hinge permits. The mechanism retains the desired stiffness in other directions independently of the softness of the spring. This is the kind of sophistication one wants of a shoe if it is to measure up to the demands of modern life.

Conclusion

Ideally your leg would have a small lever (304·8mm long) hinged onto the bottom of it. The tip of the lever would contact the ground first, and as your weight came on to that leg, it would pivot about the hinge stretching a spring to absorb the impact and lower your heel gently to the ground. If you are carrying a heavy rucksack the springs would have to be adjusted for the heavier load. Worse, if you were carrying a heavy suitcase with one hand, the springs would have to be adjusted differently and readjusted when you changed hands! So it needs to be an active spring under micro-processor control.

How much would such a pair of shoes cost? $500, $5000, who cares? You already own a pair that came free, as your body's standard equipment. The small lever is called the foot, the hinge is called the ankle, the spring is the Achilles tendon, the adjustment and damping is provided by the calf muscle. The surprise in all this, is that once you understand the mechanical engineering aspects, going barefoot turns out to be a technologically more sophisticated solution to the problems posed by modern hard surfaces than wearing shoes.

The transition to going barefoot is hard. You need to get you eye in for spotting broken glass. You need to sharpen up your foot-eye co-ordination, so you can avoid it once you have seen it. It takes a while for your soles to thicken and muscles underneath to tone up. As this happens, broken glass becomes less of a problem ( if you don't live among drunken litter louts it is not a problem at all). It takes some months to get your calf muscles toned up and to learn to use them correctly. You have to place your foot, not scuff it; as though you were reaching forward to grab the pavement with your toes and pull it back underneath you.

The payoff for all this effort is wonderful. You literally get a spring in your step. Walking becomes a pleasure, like dancing, instead of being a misfortune endured when your car breaks down. You can use the new strength in your ankles to rise up a couple of inches when climbing stairs. Steep stairs become shallow and you feel twenty years younger.

Is there anything I want to add in 2024? Yes, a subtle point about geometry. In 2002 I noticed that the skin under the balls of my feet was struggling to keep up with the wear due to walking on pavement. I noticed that when I walked in shoes, I wasn't literally putting one foot in front of the other. The right foot would be placed in front of where the right foot had been. The left foot would be placed in front of where the left foot had been. But the two feet followed parallel tracks about 9 inches apart. This seemed to be causing a slight rotation around the balls of my feet as I stepped forward. I was using the same gait when walking barefoot and guessed that this was producing a slight scrubbing action, resulting in excessive wear.

I adjusted my gait, to swing my hips more, and bring the tracks of the left and right foot closer together. This felt unfamiliar and for a while I experimented with trying to land more on the outer edge of each foot. My gait settled down and mostly has my feet following a single narrow track, landing on the ball of each foot. This solved the problem of excessive skin wear. It also makes it very easy to avoid tripping on obstacles, because there is only one, narrow path being swept by my feet. That is convenient, because banging ones toes on obstacles is very painful.

My 2024 addition is partly prompted by the tag "Self Experimentation". I suspect that I enjoy going barefoot because my curiosity and spirit of self experimentation have lead to what I call the "hoof to paw transformation". Feeling different textures is part of the fun. I see textures ahead and adjust my path of land on them. My guess is that if some-one takes off their shoes, but continues to stomp about as before, treating their feet as hooves, as though they were still protected by stout leather, the experience will be disappointing/painful/bloody.

This reminds me of a passage in Richard Feynman's memoir "What do you care what other people think?". Four pages into the chapter Gumshoes, (page 163 in the Unwin Paperback edition):

Then this business of Thiokol changing its position came up. Mr. Rogers and Dr. Ride were asking two Thiokol managers, Mr. Mason and Mr. Lund, how many people were against the launch, even at the last moment.

"We didn't poll everyone," says Mr. Mason.

"Was there a substantial number against the launch, or just one or two?"

"There were, I would say, probably five or six in engineering who at that point would have said it is not as conservative to go with that temperature, and we don't know. The issue was we didn't know for sure that it would work."

"So it was evenly divided?"

"That's a very estimated number."

It struck me that the Thiokol managers were waffling. But I only knew how to ask simpleminded questions. So I said, "Could you tell me, sirs, the names of your four best seals experts, in order of ability?"

"Roger Boisjoly and Arnie Thompson are one and two. Then there's Jack Kapp, and, uh ... Jerry Burns."

I turned to Mr. Boisjoly, who was right there, at the meeting. "Mr. Boisjoly, were you in agreement that it was okay to fly?"

He says, "No, I was not."

I ask Mr. Thompson, who was also there.

"No. I was not."

I say "Mr. Kapp?"

Mr. Lund says, "He is not here, I talked to him after the meeting, and he said, 'I would have made that decision, given the information we had.'"

"And the fourth man?"

"Jerry Burns. I don't know what his position was."

"So," I said, "of the four, we have one 'don't know,' one 'very likely yes,' and the two who were mentioned right away as being the best seal experts, both said no." So this "evenly split" stuff was a lot of crap. The guys who knew the most about the seals --- what were they saying?

That is the end of that section of the chapter and Feynman turns to the infra-red thermometer and the temperatures on the launch pad.

That was my introduction to this aspect of bureaucratic infighting. The bureaucrat asks his technical experts, the one closest to the issue. If he gets the answer that he wants, it is accepted. If not, he widens the pool of experts. Those too close to the issue are at risk of ignoring the social cues to the desired answer, but the wider pool of experts can be more flexible at responding to the broader social context. Then the bureaucrat gets to take an unweighted average (that is not weighting the original experts more highly). Which boosts the probability of getting the desired answer and reduces the probability of getting the correct answer.

Back in 1988 this was perhaps a busted technique. But that was many years ago. The notion of broadening your survey of experts seems to be back in fashion.

From the perspective of 2023, censorship looks old fashioned; new approaches create popular enthusiasm around government narratives.

For example, the modern way for the Chinese to handle Tiananmen Square is to teach the Chinese people about it, how it is an American disinformation campaign that aims to destabilize the PRC by inventing a massacre that never happened, and this is a good example of why you should hate America.

Of course there are conspiracy theorist who say that it actually happened and the government covered it up. What happened to the bodies? Notice that the conspiracy theorists are also flat Earthers who think that the PRC hid the bodies by pushing them over the edge. You would not want to be crazy like them, would you?

Then ordinary people do the censorship themselves, mocking people who talk about Tiananmen Square as American Shills or Conspiracy Theorists. There is no need to crack down hard on grumblers. Indeed the grumblers can be absorbed into the narrative as proof that the PRC is a kindly, tolerant government that permits free speech, even the worthless crap.

I don't know how LLM's fit into this. Possibly posting on forums to boost the official narrative. Censorship turns down the volume on dissent, but turning up the volume on the official narrative seems to work better.