This is a special post for quick takes by Max Harms. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
IABI says: "Transistors, a basic building block of all computers, can switch on and off billions of times per second; unusually fast neurons, by contrast, spike only a hundred times per second. Even if it took 1,000 transistor operations to do the work of a single neural spike, and even if artificial intelligence was limited to modern hardware, that implies human-quality thinking could be emulated 10,000 times faster on a machine— to say nothing of what an AI could do with improved algorithms and improved hardware.
@EigenGender says "aahhhhh this is not how any of this works" and calls it an "egregious error". Another poster says it's "utterly false."
(Relevant online resources text.)
(Potentially relevant LessWrong post.)
I am confused what the issue is, and it would be awesome if someone can explain it to me.
Where I'm coming from, for context:
- We don't know exactly what the relevant logical operations in the human brain are. The model of the brain that says there are binary spiking neurons that have direct connections from synapse->dendrite and that those connections are akin to floating-point numerical weights is clearly a simplification, albeit a powerful one. (IIUC "neural nets
1
1The passage seems fine to me; I commented on Erdil's post and other brain efficiency discussions at the time, and I still think that power consumption is a more objective way of comparing performance characteristics of the brain vs. silicon, and that various kinds of FLOP/s comparisons favored by critics of the clock speed argument in the IAB passage are much more fraught ([1], [2]).
It's true that clock speed (and neuron firing speed) aren't straightforwardly / directly translatable to "speed of thought", but both of them are direct proxies for energy consumption and power density. And a very rough BOTEC shows that ~10,000x is a reasonable estimate for the difference in power density between the brain and silicon.
Essentially, the brain is massively underclocked because of design-space restrictions imposed by biology and evolution, whereas silicon-based processing has been running up against fundamental physical limits on component size, clock speed, and power density for a while now. So once AIs can run whatever cognitive algorithms that the brain implements (or algorithms that match the brain in terms of high-level quality of the actual thoughts) at any speed, the already-existing...
1
18
My response.
[...]
Good point actually, and yeah the ability to move information from one area to the other much faster than brains do is arguably why NNs make different tradeoffs than human brains.
[...]
For what it's worth, I wasn't depending on the premise that a synapse is computationally more powerful than an artificial neuron.
[...]
I actually agree with this take, but critically not in the domain of allowing AIs to think faster, which was my original objection.
@Alexander Gietelink Oldenziel and @S. Alex Bradt and @Max Harms: The thing I was talking about when I agreed with the claim that the brain is more powerful and doing fancier things is basically the fact that the brain always learns and thinks, called continual learning or continual thinking (there is no knowledge cutoff for brains the way current LLMs have), as well as better long-term memory/keeping things in context.
I do agree that in general, human brains aren't too special algorithmically.
Here are some links as to why below:
lc on why the current lack of long-term memory creates problems, especially for benchmarking
Dwarkesh Patel and Gwern on continual learning/thinking.
[...]
Do we actually have a source for this, or is this just a commonly believed fact about AIs? I'm getting worried that this claim isn't actually supported by much evidence and is instead a social belief around AIs due to our previous prediction errors.
I do think AIs can run quite a bit faster than humans, I'm just making the claim that the transistor argument is locally invalid.
Edit: @Max Harms I no longer endorse this objection, and now think my comment on it being utterly false that AI thinking speeds would increase drastically was not correct, and Max H explains why.
6
Josh You has some interesting points on this subject. Quoting him for any counterarguments, as well as responses:
[...]
6
If you have a lump of 7,000 neurons, they can each connect to each other neuron, and you can spherical-cow approximate that as a 7000x7000 matrix multiplication. That matrix multiplication will all happen within O(1) spikes, 1/100 of a second. That's ~700GFlop. An H100 GPU takes ~1 millisecond to do that operation, or 1M cycles, to approximate one brain spike cycle! And the gpu has 70B or whatever transistors, so it's more like 10M transistors per neuron!
6
I'm guessing the intended meaning of "modern hardware" in the quote is easy to misread (it's actually just gesturing at the transistors), and for many unintended senses of "modern hardware" (actual AI chips and systems, actual LLMs with KV cache) the claims in the quote are obviously wrong.
4
* The MoE architecture doesn't just avoid thrashing weights around. It also reduces the amount of calculations per token. For instance, DeepSeek v3.1 has 671B parameters, out of which 37B are activated per token and used in matrices. A model like GPT-3 would use all the 175B parameters it has.
* IIRC the human brain makes 1E14 -- 1E15 FLOP/second. The authors of the AI-2027 forecast imply that a human brain creates ~10 tokens/sec, or uses 1E13 -- 1E14 computations per token while having 1E14 synapses.
A more detailed analysis of Yudkowsky's case for FOOM
If the brain was magically accelerated a million times so that signals reached the speed of 100 million m/s, then the brain would do 1E20 -- 1E21 FLOP/second while doing 1E17 transitions/sec. Cannell's case for brain efficiency claims that the fundamental baseline irreversible (nano) wire energy is: ~1 Eb/bit/nm, with Eb in the range of 0.1eV (low reliability) to 1eV (high reliability). If reliability is low and each transition is 1E7 nanometers or 1 centimeter, then we need 1E23 EV/second or 1E4 joules/second. IMO this implies that Yudkowsky's case for a human brain accelerated a million times is as unreliable as Cotra's case against AI arriving quickly. However, proving that AI is an existential threat is far easier since it requires us to construct an architecture, not to prove that there's none.
* Returning to the human brain being far more powerful or efficient, we notice that it can't, say, be copied infinitely many times. If it could, one could, say, upload a genius physicist and have an army of its copies work on different projects and exchange insights.
* As for the humans being "wildly more data efficient", Cannell's post implies that AlphaGo disproves this conjecture with regards to narrow domains like games. What the humans are wildly more efficient is their ability to handle big contexts and to keep the information in mind for more than a single forward pass, as I discussed here and in the
2
on the bio side, I'd imagine the transistor equivalent is not a neuron spike, but rather a protein binding
5
Excellent. Thanks for the thoughtful response.
[...]
I picked Parfit's Hitchhiker (PH) because it's the result that I think maps most cleanly to real-world moral dilemmas, but I think we can probably make headway using fairness and cooperation results (Ultimatum Game and Prisoner's Dilemma) rather than reciprocity/character (PH). If we're getting stuck it might be worth diving into why you're unsure about Newcomb, but I'll leave it for now.
[...]
Yep. This is a valid way to read my claim.
[...]
Those look to me like the same statement. Do you see "ought" and "should" as non-synonyms?
[...]
This introduces a new thing -- whether a person can be good or bad -- which is distinct from the argument that I'm trying to make in favor of moral realism. I have a perspective on this question, if it feels vital to you, but I'd rather stick to whether actions can have moral character.
[...]
Why not? What's missing?
[...]
Troll: "I got one wolf and one hare, and put them in a pen together and now I have one wolf. I have found evidence that 1+1=1."
More seriously, I think your stance is fine. I tend to use the correspondence theory of truth, where a model of reality is true insofar as the structure of the model matches the structure of reality. I tend to see math as an abstract structure that can be reasoned about in absence of any application to reality (and thus any notion of truth). Then, we can apply that structure to specific situations to get a more concrete model, which can be true or false, but it's not always correct to apply a certain bit of math.
But I'm happy to use a more Platonic frame where the abstraction can be true in isolation. I just wanted to check.
[...]
Noted. I'll try to aim towards real-world observations.
[...]
Troll: "Lots of smart people describe themselves as moral realists, and even, like Max, become more aligned with moral realism over time as smart adults (instead of being indoctrinated as kids). This is more likely to be observed
8
Sure, "should" and "ought" are synonyms, but "you [should/ought to] do X because it's instrumentally useful" and "you [should/ought to] do X because it's moral" are very different claims.
5
Sure, sorry, didn't mean to expand the scope there. Let's rephrase it to "and you're doing something bad if you don't".
(Although if you convince me that actions can be objectively morally bad, it's pretty trivial to convince me that people who do those actions can be objectively morally bad.)
3
Either of these approaches are fine; I don't think you have any evidence for moral realism either way.
(I too mostly use the correspondence theory of truth; sorry if I implied I think it's wrong)
3
"It would be instrumentally beneficial for you to do X" and "It would be morally good for you to do X" are totally different claims! (Unless you're telling me it's always instrumentally good to behave morally, which seems implausible unless there's a god who sends immoral people to hell after they die.)
1
You're joking but I really do consider the bolded sentence to be evidence for moral realism -- in fact, it's the strongest evidence for moral realism I've been able to find.
But there are many better explanations for lots of smart people believing a false thing, that have much more evidence supporting them, than moral realism actually being true.
1
That's just not what "moral facts" means. Theorems of game theory can be facts and they can be relevant to what you should do in certain situations where you might also consider moral theories, but they aren't moral claims. A moral claim is something like "murder is wrong" or "slavery is acceptable". On the other hand, "murder is illegal" or "slavery was legal in the Roman Empire and the Confederate States of America" are not the kind of moral facts I'm talking about, and they (and game theory theorems) don't prove/disprove moral realism.
8
I'm trying to get you to define what "moral fact" is, and contrast it with the normative statements that I can prove, so that we can move past semantics. It's my understanding that the literature uses the term to refer to something like categorical imperatives -- pressures to do certain things that apply to all reasoning beings, regardless of who they are.
They can't be infinitely strong pressures since obviously people sometimes do bad things, if there are bad things, but they have to have some motive force where they bind into the agent's motivational system. And since they're universal, they can't bind to the agent's terminal values. Thus, I claim, in order for a categorical imperative to make any sense, we must be able to see it show up in convergent instrumental strategy/subgoals.
Are all instrumentally convergent subgoals categorical imperatives ("moral facts")? I don't think so. From my perspective, a defining characteristic of the thing that I want to look at is that the pressure comes from the social fabric. The standard Omohundro drives of survival, rationality, resource acquisition, etc, are all just as applicable on a deserted island. But I claim that once you posit other agents, even if they're powerless or distant or hypothetical, then there are arguments in the same space as "don't defect against your clone your dumbass" that come into play.
What I'd really like is for you to taboo "moral fact" or "moral" or "wrong" or whatever and try saying the equivalent of "Theorems of game theory are absolutely not moral facts."
Here's how it caches out it my semantics: "Theorems of game theory are absolutely not [facts that bear on how any agent, regardless of who they are, ought to act in certain classes of situations, according to their own values, because of the acausal interplay (including the establishment of Schelling points, red lines, and other coordination mechanisms) between that agent and the other agents in the universe (including distant, causal
4
I wonder how similar Robi's view on morality is to the idea that a fairly close equivalent of moral facts irreducible to game theory is the statement about consequences which are hard to trace or to explain.
Suppose that polyamory is net negative for most people who try it because such relationships bring genuine happiness to those who possess a rare quality and are fragile unless the participants possess the quality. IMO under these conditions Aella would expect people to be thoroughly tested to determine whether they possess the right qualities, then taking measures to ensure that incompatible people don't do things like wasting time with each other. However, I expect such measures to be bottlenecked on things like it being hard to explain to participants why they would be unlikely to succeed, so the simplest measure in a society of those whom we don't trust to be geniuses is to simply ban polyamory wholesale.
Similarly, if we trust the rumors that gentle-parenting-as-millenials-describe-it is hard to pull off in a manner which doesn't produce a misaligned kid, then, once a cohort of kids parented that way emerges and is studied, the ideal world would have parents avoid this practice unless it is verified that they can succeed at gently raising the kid, and a world like ours would also require that parents don't erroneously believe that they can.
However, my[1] example of WIMOPs hating hybrids would have Yudkowsky's case against morality being given ensure that I, unlike Robi, don't follow the WIMOPs' anti-hybrid stance unless the hybrids are degrading in ways which are proven, but hard to explain to the commoner.
1. ^
To be precise, the example was actually coined by a Russian extremist author, but the extremist had the fictional hybrids and their descendants actually degrade.
3
I agree, theorems of game theory are extremely relevant to that thing you described.
[...]
Again, I googled "what is morality" and the default result was pretty good. It said:
[...]
1
I'm not a moral relativist; I don't believe that what's right or wrong is decided by the beliefs of society or the context in which the action is taken. I believe there is no truth value to whether actions are objectively right or wrong.
1
I just googled "what are moral facts" and its answer is pretty good:
[...]
I would add:
Moral facts are moral claims that are true.
My position is that there are no moral claims that have any truth values: none of them are true and none of them are false. They are not truth-apt; they are not in the category of things that can be true or false.
Regarding your game theory example, your claim that (because being the kind of agent who cooperates in the prisoner's dilemma will benefit you in the long run) "you should cooperate in the prisoner's dilemma" (where "should" means "is in your best interest/increases your expected utility") is not a moral claim of the kind I am talking about. The claim that "you should cooperate in the prisoner's dilemma", where "should" means "it is morally good to cooperate", is a moral claim, but is neither true nor false. I'm asking for any evidence that it, or any other moral claim, is true or false.
4
Yeah, I donno. This doesn't feel like you're tabooing things correctly. Imagine I put on the alien hat and pretend I don't know these silly earth words. Tell me what morality is.
[...]
Sure. I claim "people should cooperate against their clones in PD" is a statement about what people ought to do. Is that an [UNTRANSLATABLE-1] fact? Why not?
[...]
The first and second examples are presumably things that are illegal, antisocial, and violations of the sorts of red-lines that can be drawn behind the veil of ignorance, among many other things. What makes them [UNTRANSLATABLE-3] and [UNTRANSLATABLE-5]? Are [UNTRANSLATABLE-3] and [UNTRANSLATABLE-5] synonyms? Do the [UNTRANSLATABLE-8] and [UNTRANSLATABLE-9] modifiers matter?
[...]
Sure. That's basically what "fact" means.
[...]
Wait. If [UNTRANSLATABLE-1] claims are not in the category of things that can be true, then how can your WIMP example work? Didn't you say you would change your mind if we detected WIMPs? How is this not in contradiction with the notion that it's categorically not something that can be true?
[...]
Cool. I hear you that you don't consider categorical imperatives to be [UNTRANSLATABLE-1] facts. But I still don't know what you mean by [UNTRANSLATABLE-1]. Help! 👽
Defining morality while tabooing moral, ethical, right, wrong, good, bad, ought, should, duty, obligation, permissible, virtue, and cognate words:
Humans sometimes classify actions, intentions, people, institutions, and outcomes using positive, negative, or neutral labels that supposedly do more than describe what happened, predict consequences, report a law or custom, state what anyone wants, or identify an effective means to a chosen end.
Central examples of things that receive these classifications concern causing pain, killing, deception, coercion, keeping promises, helping people in distress, and distributing burdens and benefits. They contrast with judgments about which chess move wins, which investment earns more money, which action is legal, which behavior is fashionable, or which policy best fulfills a specified utility function.
The disputed question is whether some applications of these labels accurately represent features or relations that are not made accurate merely by anyone’s preferences, commitments, conventions, or decisions. My position is that applying the labels expresses attitudes; it does not report an additional feature of the action.
By "morality" I mean the hu...
13
(Sorry, I hadn't attempted the taboo exercise yet, I was trying one more time at just giving the straightforward definition. Might retry but with all morality words tabooed, if I have time.)
3
Sorry. Clarifying: they are not things that it is categorically impossible for them to be true; they are things that have no evidence for their truth-aptness. If WIMPs were discovered then I would realize they are indeed in the category of truth-apt things.
1
"people should cooperate against their clones in PD" is a statement about what people ought to do, where "ought" means "is in one's self-interest". It is not a statement about [UNTRANSLATABLE-1] because statements about what is in one's self-interest are not the category of things that [UNTRANSLATABLE-1] is referring to.
3
I don't think I can convincingly argue for moral realism, but IMO you are too quick to dismiss it. There is extensive philosophical literature addressing your objections (as I understand them). I'm not saying that any of that literature is definitely correct, but you seem to be under-rating it. I object to your acting as if meta-ethics is basically solved and saying (in jest) that it's confusing when smart people are realists, while also not really addressing any of the arguments in favor of realism.
For example:
[...]
I've never heard a moral realist claim that their position can be supported experimentally.
Or:
[...]
The evolutionary argument against intuition has been addressed by e.g. Parfit in On What Matters. I'm not sure his defense is correct, but you can't claim that Parfit simply doesn't understand evolution.
62% of philosophers are realists (and 65% of metaethicists); it sounds like you are advocating for expressivism, which only 11% of philosophers agree with.
I point to surveys rather than making any object-level argument because I don't think I can make good arguments on meta-ethics, you wouldn't learn much from my arguments, and the arguments have to go for a long time to get anywhere at all. Instead, I'm pointing to the fact that a lot of literature exists that makes in-depth arguments for why realism is true.
6
As a practicing Catholic and moral realist, something like what Max wrote above, represents my best current understanding of how to reason about moral truth, that is: a set of rules that governs how game theory agents ought to interact.
3
I think maybe it wouldn't be "All agent should do X", because you can have agents with weird values that reward not X for anything including cooperating with themselves etc.
So then it's a somewhat different thing to scientific facts if for any moral statement you can say "no I don't care, sure this points to real structure about other agents, but it doesn't apply to me" and be correct. Noone can say "this doesn't apply to me" about the laws of physics and be correct.
Though if you assume agents with coherent values then the statements about decision theory etc kind of just boil down to "you should fulfill your values" and that applies to weird values as well, so they are either incoherent or the statements apply.
"You should fulfill your values" and variants of it are a pretty small statement set though so effectively most moral statements like "murder is wrong" or "There should be more pleasure" end up subjective.
3
Correct me if I'm wrong but based on the types of things you mentioned above, it seems like you might be misunderstanding what moral facts are. For example, you might argue:
* If game theory and decision theory indicate that you get better payoffs from cooperating than defecting, you should cooperate.
* Game theory and decision theory indicate that you get better payoffs from cooperating than from defecting. [insert various arguments/proofs here]
* Therefore, you should cooperate.
The last line, if interpreted as "you get better payoffs from cooperating" would be true, while "it's morally good to cooperate and morally bad to defect" would be a moral fact if true, but it's still an opinion -- this type of argument hasn't provided evidence for it.