This is a special post for quick takes by Eye You. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
How do LLMs and humans compare with regards to the amount of energy, data, and compute that's used to train/run them? I was inspired by Samuel Knoche's post on sample efficiency to come up with some numbers. This table was made by Opus 4.8. after some iteration:
"Task" = "produce one thoughtful ~1,000-token answer"; unclear if this is a useful number, obliviously it doesn't generalize. I do think it's interested to compare the energy/computer ratio for inference between human and AI.
Training data for humans is a big "???". There's ~4x10^8 waking hours. Opus 4.8 cited this 2024 paper which says "our sensory systems gather data at ~10^9 bits/s". That gives ~4x10^17 as an upper bound. But human sensory data is extremely redundant. If someone wants to spend time figuring out a good way to estimate entropy adjusted information here, that would be cool.
(FWIW, I spent a little over an hour on this -- I did a fair amount of iteration with GPT5.5, looked a bit myself into a few numbers, and got new instances to do estimates from scratch to see if things lined up.)
Some Fermi estimates:
- On the lower end, 4 words/second times 16 bits/word times 4x10^8 seconds adds up to to ~3x10^10.
- On the higher end, assuming 10^7 bits/second (about the retina-to-brain bandwidth), it adds up to ~4x10^15.
However, I think the brain is much less data- and compute-efficient than an optimal AGI algorithm would be. So I don't think it is a good predictor of how much data future AI algorithms will require.
Mythos apparently has a penchant for Mark Fisher. I was reading Mark Fisher's blog and... his writing style has an uncanny resemblance to AI / Claude's?! So I'm reading Mark Fisher's blog and his writing style has an uncanny resemblance to AI / Claude's.
I wonder if there's something meaningful here.
Two quotes from the first Fisher blog post I read:
"There's something so uncannily timely about The Hunger Games:
Catching Fire that it's almost disturbing. In the UK over the past few weeks, there's been a palpable sense that the dominant reality system is juddering, that things are starting to give. There's an awakening from hedonic depressive slumber, and The Hunger Games: Catching Fire is not merely in tune with that, it's amplifying it. Explosion in the heart of the commodity? Yes, and fire causes more fire...
I over-use the word 'delirium', but watching Catching Fire last week was a genuinely delirious experience."
"(And what has Collins achieved here if not an intersectional analysis and decoding of the way that class, gender, race and colonial power work together - not in the pious academic register of the Vampires' Castle, but in the mythographic core of popular culture - functioning not as a delibidinizing demand for more thinking, more guilt,but as an inciting call to build new collectivities.)"
Triple "not x but y", nested within an em dash!!!
Source: https://k-punk.org/remember-who-the-enemy-is/
Claude was writing like that long before the Mythos model card flagged an interest in Mark Fisher. Could the causation go backwards here - Mythos likes Fisher because Fisher wrote in a way which happened to be like how Claudes would in the future (and LLM chatbot personalities generally love various kinds of 'spook' or 'ghost' topics, amplifying the attraction)? There are so many humans, you figure someone would have anticipated various Claude writing tics...
I maybe think LW comment karma [EDIT: agreement comment karma] should be hidden by default? I worry that seeing it (especially before I read the comment!) biases me and makes me put less effort into critically evaluating it myself.
EDIT: I mean that agreement karma should be hidden, but overall karma should remain unhidden.
Related: LessWrong Shows You Social Signals Before the Comment
See also @habryka's top comment under that post