A Cautionary Tale about Trusting Numbers from Wikipedia

So this morning I woke up early and thought to myself: "You know what I haven't done in a while? Good old fashioned Wikipedia rabbit hole." So I started reading the article on rabbits. Things were relatively sane until I got to the section on rabbits as food.

Wild leporids comprise a small portion of global rabbit-meat consumption. Domesticated descendants of the European rabbit (Oryctolagus cuniculus) that are bred and kept as livestock (a practice called cuniculture) account for the estimated 200 million tons of rabbit meat produced annually.[161] Approximately 1.2 billion rabbits are slaughtered each year for meat worldwide.[162]

Something has gone very wrong here!

200 million tons is 400 billion pounds (add ~10% if they're metric tons, but we can ignore that.)

Divide that by 1.2 billion, and we can deduce that those rabbits weigh in at over 300 pounds each on average! Now I know we've bred some large animals for livestock, but I'm rolling to disbelieve when it comes to three hundred pound bunnies.

Of the two sources Wikipedia cites it looks like [161] is the less reliable looking one. It's a WSJ blog. But the biggest reason we shouldn't be trusting that article is that its numbers aren't even internally consistent!

From the article:

Globally, about some 200 million tons of rabbit meat are produced a year, says Luo Dong, director of the Chinese Rabbit Industry Association. China consumes about 30% of the whole production, he said, with 70% of such meat—or some 420,000 tons a year—going to Sichuan province as well as the neighboring municipality of Chongqing.

There's a basic arithmetic error here! `200 million * 70% * 30%` is 42 *million*, not 420,000.

If we assume this 200 million ton number was wrong and the 420,000 ton number for Sichuan was right, the global number should in fact be 2 million tons. This would make the rabbits weigh three pounds each on average, which is a much more reasonable weight for a rabbit!

If I had to take a guess as to how this mistake happened, putting on my linguist hat, Chinese has a single word for ten thousand, like the Greek-derived "myriad", (spelt either 万 or 萬). If you actually wanted to say 2*10^6 in Chinese, it would end up as something like "two hundred myriad".  So I can see a fairly plausible way a translator could mess up and render it as "200 million".

Anyway, I've posted this essay to the talk page and submitted an edit request. We'll see how long it takes Wikipedia to fix this.

Links:
Original article: https://en.wikipedia.org/wiki/Rabbit#As_food_and_clothing 

[161] https://web.archive.org/web/20170714001053/https://blogs.wsj.com/chinarealtime/2014/06/1 3/french-rabbit-heads-the-newest-delicacy-in-chinese-cuisine/

"Aspiring Rationalist" Considered Harmful

The "aspiring" in "aspiring rationalist" seems like superfluous humility at best. Calling yourself a "rationalist" never implied perfection in the first place. It's just like how calling yourself a "guitarist" doesn't mean you think you're Jimi Hendrix. I think this analogy is a good one, because rationality is a human art, just like playing the guitar.

I suppose one might object that the word "rational" denotes a perfect standard, unlike playing the guitar. However, we don't hesitate to call someone an "idealist" or a "perfectionist" when they're putting in a serious effort to conform to an ideal or strive towards perfection, so I think this objection is weak. The "-ist" suffix already means that you're a person trying to do the thing, with all the shortcomings that entails.

Furthermore, it appears harmful to add the "aspiring". It creates dilution. Think of what it would mean for a group of people to call themselves "aspiring guitarists". The trouble is, it also applies to the sort of person who daydreams about the adulation of playing for large audiences but never gets around to practicing. However, to honestly call yourself a "guitarist", you would have to actually, y'know, play the guitar once in a while.

While I acknowledge I'm writing this many years too late, please consider dropping the phrase "aspiring rationalist" from your lexicon.

In Defense of the Shoggoth Analogy

In reply to: https://twitter.com/OwainEvans_UK/status/1636599127902662658

The explanations in the thread seem to me to be missing the middle or evading the heart of the problem.  Zoomed out: an optimization target at level of personality.  Zoomed in: a circuit diagram of layers.  But those layers with billions of weights are pretty much Turing complete.

Unfortunately, I don't think anyone has much idea how all those little learned computations are make up said personality.  My suspicion is there isn't going to be an *easy* way to explain what they're doing.  Of course, I'd be relieved to be wrong here!

This matters because the analogy in the thread between averaged faces and LLM outputs is broken in an important way.  (Nearly) every picture of a face in the training data has a nose.  When you look at the nose of an averaged face, it's based very closely on the noses of all the faces that got averaged.  However, despite the size of the training datasets for LLMs, the space of possible queries and topics of conversation is even vaster (it's exponential in the prompt-window size, unlike the query space for the average faces which are just the size of the image).

As such, LLMs are forced to extrapolate hard.  So, I'd expect that which particular generalizations they learned, hiding in those weights, to start to matter once users start poking them in unanticipated ways.

In short, if LLMs are like averaged faces, I think they're faces that will readily fall apart into Shoggoths if someone looks at them from an unanticipated or uncommon angle.

LLMs as Hecatoncheires

HP Lovecraft's Shoggoth is a pretty good metaphor for LLMs. But if I were to nominate a runner-up for mythological creature to represent them, my temptation would be to go with the Hecatoncheires. Literally, their name translates as "the hundred-handed ones". In Greek mythology they're these monsters with a hundred hands and fifty heads that are involved in the war between the Titans and the Gods, but then go back to the underworld to hold the Titans in check. (Incidentally, there might be an analogy between this story and some of the worlds where things just barely go well.)

This idea came up when I was having a conversation with Nick Tarleton the other day about what made AI-generated code and math feel different from the human-written varieties. We had concluded that AI-generated code underused abstractions compared to human-written code, and often writes logic with a zillion messy special cases that all happen to fit together if you don't try to corral it not to.

And this was when I thought to myself: wait a minute. Maybe this isn't necessarily because they're weak. Maybe the reason that *we* habitually invent helpful abstractions all over the place to make our code and our math easier to fathom is that we're the limited ones, stuck in the sort of brain that can only keep track of 7±1 things. With that bottleneck, it stands to reason that we're forced to invent useful chunks to prevent things from getting unwieldy. But when I look at the transformer architecture, naively it seems like there would be no such bottleneck. So regardless of whether LLMs *can* invent useful new abstractions, perhaps they have far less need to, because they can hold many more things in mind at once. And that's what brought to mind the hundred-handed ones.