Curated. Augmenting human intelligence seems like one of the most important things-to-think-about this century. I appreciated this post's taxonomy.
I appreciate the made of graph of made up numbers that Tsvi made up being clearly labeled as such.
I have a feeling that this post could be somewhat more thorough, maybe with more links to the places where someone could followup on the technical bits of each thread.
As someone who spent a few years researching this direction intensely before deciding to go work on AI alignment directly (the opposite direction you've gone!), I can't resist throwing in my two cents.
I think germline engineering could do a lot, if we had the multiple generations to work with. As I've told you, I don't think we have anything like near enough time for a single generation (much less five or ten).
I think direct brain tissue implantation is harder even than you imagine. Getting the neurons wired up right in an adult brain is pretty tricky. Even when people do grow new axons and dendrites after an injury to replace a fraction of their lost tissue, this sometimes goes wrong and makes things worse. Misconnected neurons are more of a problem than too few neurons.
I think there's a lot more potential in brain-computer-interfaces than you are giving them credit for, and an application you haven't mentioned.
Some things to consider here:
Brain emulation looks closer than your summary table indicates.
Manifold estimates a 48% chance by 2039.
Eon Systems is hiring for work on brain emulation.
Manifold is pretty weak evidence for anything >=1 year away because there are strong incentives to bet on short term markets.
This is great! Everybody loves human intelligence augmentation, but I've never seen a taxonomy of it before, offering handholds for getting started.
I'd say "software exobrain" is less "weaksauce," and more "80% of the peak benefits are already tapped out, for conscientious people who have heard of OneNote or Obsidian." I also am still holding out for bird neurons with portia spider architectural efficiency and human cranial volume; but I recognize that may not be as practical as it is cool.
If there's a change to human brains that human-evolution could have made, but didn't, then it is net-neutral or net-negative for inclusive relative genetic fitness. If intelligence is ceteris paribus a fitness advantage, then a change to human brains that increases intelligence must either come with other disadvantages or else be inaccessible to evolution.
You're assuming a steady state. Firstly, evolution takes time. Secondly, if humans were, for example, in an intelligence arms-race with other humans (for example, if smarter people can reliably con dumber...
Short note: We don't need 7SDs to get 7SDs.
If we could increase the average IQ by 2SDs, then we'd have lots of intelligent people looking into intelligence enhancement. In short, intelligence feeds into itself, it might be possible to start the AGI explosion in humans.
(Just acknowledging that my response is kinda disorganized. Take it or leave it, feel free to ask followups.)
Most easy interventions work on a generational scale. There's pretty easy big wins like eliminating lead poisoning (and, IDK, feeding everyone, basic medicine, internet access, less cannibalistic schooling) which we should absolutely do, regardless of any X-risk concerns. But for X-risk concerns, generational is pretty slow.
This is both in terms of increasing general intelligence, and also in terms of specific capabilities. Even if you bop an adult on the head and make zer +2SDs smarter, ze still would have to spend a bunch of time and effort to train up on some new field that's needed for the next approach to further increasing intelligence. That's not a generational scale exactly, maybe more like 10 years, but still.
We're leaking survival probability mass to an AI intelligence explosion year by year. I think we have something like 0-2 or 0-3 generations before dying to AGI.
To be clear, I'm assuming that when you say "we don't need 7SDs", you mean "we don't need to find an approach that could give 7SDs". (Though to be clear, I agree with that in a literal sense, because you...
ditto
we have really not fully explored ultrasound and afaik there is no reason to believe it's inherently weaker than administering signaling molecules.
I think you're underestimating meditation.
Since I've started meditating I've realised that I've been much more sensitive to vibes.
There's a lot of folk who would be scarily capable if the were strong in system 1, in addition to being strong in system 2.
Then there's all the other benefits that mediation can provide if done properly: additional motivation, better able to break out of narratives/notice patterns.
Then again, this is dependent on their being viable social interventions, rather than just aiming for 6 or 7 standard deviations of increase in intelligence.
Meditation has been practiced for many centuries and millions practice it currently.
Please list 3 people who got deeply into meditation, then went on to change the world in some way, not counting people like Alan Watts who changed the world by promoting or teaching meditation.
I think there are many cases of reasonably successful people who often cite either some variety of meditation, or other self-improvement regimes / habits, as having a big impact on their success. This random article I googled cites the billionaires Ray Dalio, Marc Benioff, and Bill Gates, among others. (https://trytwello.com/ceos-that-meditate/)
Similarly you could find people (like Arnold Schwarzenegger, if I recall?) citing that adopting a more mature, stoic mindset about life was helpful to them -- Ray Dalio has this whole series of videos on "life principles" that he likes. And you could find others endorsing the importance of exercise and good sleep, or of using note-taking apps to stay organized.
I think the problem is not that meditation is ineffective, but that it's not usually a multiple-standard-deviations gamechanger (and when it is, it's probably usually a case of "counting up to zero from negative", as TsviBT calls it), and it's already a known technique. If nobody else in the world meditated or took notes or got enough sleep, you could probably stack those techniques and have a big advantage. But alas, a lot of CEOs and other top performers already know to do this ...
To compare to the obvious alternative, is the evidence for meditation stronger than the evidence for prayer? I assume there are also some religious billionaires and other successful people who would attribute their success to praying every day or something like that.
Maybe other people have a very different image of meditation than I do, such that they imagine it as something much more delusional and hyperreligious? Eg, some religious people do stuff like chanting mantras, or visualizing specific images of Buddhist deities, which indeed seems pretty crazy to me.
But the kind of meditation taught by popular secular sources like Sam Harris's Waking Up app, (or that I talk about in my "Examining The Witness" youtube series about the videogame The Witness), seems to me obviously much closer to basic psychology or rationality techniques than to religious practices. Compare Sam Harris's instructions about paying attention to the contents of one's experiences, to Gendlin's idea of "Circling", or Yudkowsky's concept of "sit down and actually try to think of solutions for five minutes", or the art of "noticing confusion", or the original Feynman essay where he describes holding off on proposing solutions. So it's weird to me when people seem really skeptical of meditation and set a very high burden of proof that they wouldn't apply for other mental habits like, say, CFAR techniques.
I'm not like a meditation fanatic -- personally I don't even meditate ...
I don't understand. The hard problem of alignment/CEV/etc. is that it's not obvious how to scale intelligence while "maintaining" utility function/preferences, and this still applies for human intelligence amplification.
I suppose this is fine if the only improvement you can expect beyond human-level intelligence is "processing speed", but I would expect superhuman AI to be more intelligent in a variety of ways.
Is "give the human a calculator and a scratchpad" not allowed in this list? i.e. if you give a human brain the ability to instantly recall any fact and solve any math problem (by connecting the human brain to a computer via neuralink) seems like this would make us smarter.
We already see this effect in part. For example, having access to chatGPT allows me to program more complicated projects because I can offload sub-problems to the AI (thereby freeing up working-memory to focus on the remaining complexity). Even just having a piece of paper I c...
Questions I have:
Thanks for writing this amazing overview!
Some comments:
I read some years ago that average IQ of kids is approximately 0.25*(Mom IQ + Dad IQ + 2x population mean IQ). So simplest and cheapest means to lift population average IQ by 1 standard deviation is just use +4 sd sperm (around 1 in 30000), and high IQ ova if you can convince enough genius women to donate (or clone, given recent demonstration of male and female gamete production from stem cells). +4sd mom+dad = +2sd kids on average. This is the reality that allows ultra-wealthy dynasties to maintain ~1.3sd IQ average advantage over genera...
I think I'm more optimistic about starting with relatively weak intelligence augmentation. For now, I test my fluid intelligence at various times throughout the day (I'm working on better tests justified by algorithmic information theory in the style of Prof Hernandez-Orallo, like this one but it sucks to take https://github.com/mathemajician/AIQ but for now I use my own here: https://github.com/ColeWyeth/Brain-Training-Game), and I correlate the results with everything else I track about my lifestyle using reflect: https://apps.apple.com/ca/app/reflect-tr...
I think it makes sense to pick the low-hanging fruit first (then attempt incrementally harder stuff with the benefit of being slightly smarter)
No, this doesn't make sense.
I think the stuff you're doing is probably fun / cool / interesting / helpful / something you like. That's great! You don't need to make an excuse for doing it, in terms of something about something else.
But no, that's not the right way to make really smart humans. The right way is to directly create the science and tech. You're saying something like "it stands to reason that if we can get a 5% boost on general intelligence, we should do that first, and then apply that to the tech". But
as I understand it, the AI capabilities necessary for Intelligence amplification via BCI already exist, and we simply need to show/encourage people how to start using it
If a person were to provide a state-of-the-art model with a month's worth of their data typically collected by our eyes and ears and the ability to interject in real time in conversations via earbuds or speaker.
Such an intervention wouldn't be the superhuman "team of geniuses in your datacenter" but it would be more helpful than even some of the best personal assistant's (and 10...
On human-computer interfaces: Working memory, knowledge reservoirs and raw calculation power seem like the easiest pieces, while fundamentally making people better at critical thinking, philosophy or speeding up actual comprehension would be much for difficult.
The difference being upgrading the core vs plug-ins.
Curated reservoirs of practical and theoretical information, well indexed, would be very useful to super geniuses.
On human-human: You don't actually need to hook them up physically. Having multiple people working on different parts of a problem let...
I feel like it would be beneficial to add another sentence or two to the “goal” section, because I’m not at all convinced that we want this. As someone new to this topic, my emotional reaction to reading this list is terror.
Any of these techniques would surely be available to only a small fraction of the world’s population. And I feel like that would almost certainly result in a much worse world than today, for many of the same reasons as AGI. It will greatly increase the distance between the haves and the have-nots. (I get the same feeling reading this as...
The genetic portions of this seem like a manifesto for creating highly intelligent, highly depressed, and thus highly unproductive people.
I made up the made-up numbers in this table of made-up numbers; therefore, the numbers in this table of made-up numbers are made-up numbe
These hallucinated outputs are really getting out of hand
I think you're underestimating meditation.
Since I've started meditating I've realised that I've been much more sensitive to vibes.
There's a lot of folk who would be scarily capable if the were strong in system 1, in addition to being strong in system 2.
Then there's all the other benefits that mediation can provide if done properly: additional motivation, better able to break out of narratives/notice patterns.
Thanks for the detailed writeup. I would personally be against basically all of the suggested methods that could create a significant improvement because the hard problem of consciousness remains hard and it seems very possible that an unconscious human race could result. I was a bit surprised to see no mention of this in the essay.
I think youre making this more complicated than it has to be. Why try to move a river to you when you can move to the river? Social Engineering is the way, I think. The same way that flat-surfaced guardrails on stars encourage people leaving trash/drinks/whatever there so does everything else in our public life (the life that we have when interacting with others- going to the store; filling with gas; waiting in lines; shopping etc). Combining microhabits with de-atrophication of our brains is the easiest and most widely possible solution. Maybe create a pr...
Somewhat surprised that this list doesn't include something along the lines of "punt this problem to a sufficiently advanced AI of the near future." This could potentially dramatically decrease the amount of time required to implement some of these proposals, or otherwise yield (and proceed to implement) new promising proposals.
It seems to me in general that human intelligence augmentation is often framed in a vaguely-zero-sum way with getting AGI ("we have to all get a lot smarter before AGI, or else..."), but it seems quite possible that AGI or near-AGI could itself help with the problem of human intelligence augmentation.
I don't think you mentioned "nootropic drugs" (unless "signaling molecules" is meant to cover that, though it seems more specific). I don't think there's anything known to give a significant enhancement beyond alertness, but in a list of speculative technologies I think it belongs.
How can we make many humans who are very good at solving difficult problems?
Summary (table of made-up numbers)
I made up the made-up numbers in this table of made-up numbers; therefore, the numbers in this table of made-up numbers are made-up numbers.
Call to action
If you have a shitload of money, there are some projects you can give money to that would make supergenius humans on demand happen faster. If you have a fuckton of money, there are projects whose creation you could fund that would greatly accelerate this technology.
If you're young and smart, or are already an expert in either stem cell / reproductive biology, biotech, or anything related to brain-computer interfaces, there are some projects you could work on.
If neither, think hard, maybe I missed something.
You can DM me or gmail me at tsvibtcontact.
Context
The goal
What empowers humanity is the ability of humans to notice, recognize, remember, correlate, ideate, tinker, explain, test, judge, communicate, interrogate, and design. To increase human empowerment, improve those abilities by improving their source: human brains.
AGI is going to destroy the future's promise of massive humane value. To prevent that, create humans who can navigate the creation of AGI. Humans alive now can't figure out how to make AGI that leads to a humane universe.
These are desirable virtues: philosophical problem-solving ability, creativity, wisdom, taste, memory, speed, cleverness, understanding, judgement. These virtues depend on mental and social software, but can also be enhanced by enhancing human brains.
How much? To navigate the creation of AGI will likely require solving philosophical problems that are beyond the capabilities of the current population of humans, given the available time (some decades). Six standard deviations is 1 in 10^9, seven standard deviations is 1 in 10^12. So the goal is to create many people who are 7 SDs above the mean in cognitive capabilities. That's "strong human intelligence amplification". (Why not more SDs? There are many downside risks to changing the process that creates humans, so going further is an unnecessary risk.)
It is my conviction that this is the only way forward for humanity.
Constraint: Algernon's law
Algernon's law: If there's a change to human brains that human-evolution could have made, but didn't, then it is net-neutral or net-negative for inclusive relative genetic fitness. If intelligence is ceteris paribus a fitness advantage, then a change to human brains that increases intelligence must either come with other disadvantages or else be inaccessible to evolution.
Ways around Algernon's law, increasing intelligence anyway:
How to know what makes a smart brain
Figure it out ourselves
Copy nature's work
Brain emulation
The approach
The idea is to have a human brain, but with the advantages of being in a computer: faster processing, more scalable hardware, more introspectable (e.g. read access to all internals, even if they are obscured; computation traces), reproducible computations, A/B testing components or other tweaks, low-level optimizable, process forking. This is a "figure it out ourselves" method——we'd have to figure out what makes the emulated brain smarter.
Problems
While we have some handle on the fast (<1 second) processes that happen in a neuron, no one knows much about the slow (>5 second) processes. The slow processes are necessary for what we care about in thinking. People working on brain emulation mostly aren't working on this problem because they have enough problems as it is.
Experiments here, the sort that would give 0-to-1 end-to-end feedback about whether the whole thing is working, would be extremely expensive; and unit tests are much harder to calibrate (what reference to use?).
Partial success could constitute a major AGI advance, which would be extremely dangerous. Unlike most of the other approaches listed here, brain emulations wouldn't be hardware-bound (skull-size bound).
The potential for value drift——making a human-like mind with altered / distorted / alien values——is much higher here than with the other approaches. This might be especially selected for: subcortical brain structures, which are especially value-laden, are more physiologically heterogeneous than cortical structures, and therefore would require substantially more scientific work to model accurately. Further: because the emulation approach is based on copying as much as possible and then filling in details by seeing what works, many details will be filled in by non-humane processes (such as the shaping processes in normal human childhood).
Fundamentally, brain emulations are a 0-to-1 move, whereas the other approaches take a normal human brain as the basic engine and then modify it in some way. The 0-to-1 approach is more difficult, more speculative, and riskier.
Genomic approaches
These approaches look at the 7 billion natural experiments and see which genetic variants correlate with intelligence. IQ is a very imperfect but measurable and sufficient proxy for problem-solving ability. Since >7 of every 10 IQ points are explained by genetic variation, we can extract a lot of what nature knows about what makes brains have many capabilities. We can't get that knowledge about capable brains in a form usable as engineering (to build a brain from scratch), but we can at least get it in a form usable as scores (which genomes make brains with fewer or more capabilities). These are "copy nature's work" approaches.
Adult brain gene editing
The approach
See "Significantly Enhancing ...".
Problems
Delivery is difficult.
Editors damage DNA.
The effect is greatly attenuated, compared to germline genetics. In adulthood, learning windows have been passed by; many genes are no longer active; damage that accumulates has already been accumulated; many cells don't receive the edits. This adds up to an optimistic ceiling somewhere around +2 or +3 SDs.
Germline engineering
This is the way that will work. (Note that there are many downside risks to germline engineering, though AFAICT they can be alleviated to such an extent that the tradeoff is worth it by far.)
The approach
Subtasks:
Know what genome would produce geniuses. This is already solved well enough. Because there are already polygenic scores for IQ that explain >12% of the observed variance in IQ (pgscatalog.org/score/PGS003724/), 10 SDs of raw selection power would translate into trait selection power at a rate greater than √(1/9) = 1/3, giving >3.3 SDs of IQ trait selection power, i.e. +50 IQ points.
Make a cell with such a genome. This is probably not that hard——via CRISPR editing stem cells, via iterated meiotic selection, or via chromosome selection. My math and simulations show that several methods would achieve strong intelligence amplification. If induced meiosis into culturable cells is developed, IMS can provide >10 SDs of raw selection power given very roughly $10^5 and a few months.
Know what epigenomic state (in sperm / egg / zygote) leads to healthy development. This is not fully understood——it's an open problem that can be worked on.
Given a cell, make a derived cell (diploid mitotic or haploid meiotic offspring cell) with that epigenomic state. This is not fully understood——it's an open problem that can be worked on. This is the main bottleneck.
These tasks don't necessarily completely factor out. For example, some approaches might try to "piggyback" off the natural epigenomic reset by using chromosomes from natural gametes or zygotes, which will have the correct epigenomic state already.
See also Branwen, "Embryo Selection ...".
More information on request. Some of the important research is happening, but there's always room for more funding and talent.
Problems
It takes a long time; the baby has to grow up. (But we probably have time, and delaying AGI only helps if you have an out.)
Correcting the epigenomic state of a cell to be developmentally competent is unsolved.
The baby can't consent, unlike with other approaches, which work with adults. (But the baby can also be made genomically disposed to be exceptionally healthy and sane.)
It's the most politically contentious approach.
Signaling molecules for creative brains
The approach
Due to evolved modularity, organic systems are governed by genomic regulatory networks. Maybe we can isolate and artificially activate GRNs that generate physiological states that produce cognitive capabilities not otherwise available in a default adult's brain. The hope is that there's a very small set of master regulators that can turn on larger circuits with strong orchestrated effects, as is the case with hormones, so that treatments are relatively simple, high-leverage, and discoverable. For example, maybe we could replicate the signaling context that activates childish learning capabilities, or maybe we could replicate the signaling context that activates parietal problem-solving in more brain tissue.
I haven't looked into this enough to know whether or not it makes sense. This is a "copy nature's work" approach: nature knows more about how to make brains that are good at thinking, than what is expressed in a normal adult human.
Problems
Who knows what negative effects might result.
Learning windows might be irreversibly lost after childhood, e.g. by long-range connections being irrecoverably pruned.
Brain-brain electrical interface approaches
Brain-computer interfaces don't obviously give an opportunity for large increases in creative philosophical problem-solving ability. See the discussion in "Prosthetic connectivity". The fundamental problem is that we, programming the computer part, don't know how to write code that does transformations that will be useful for neural minds.
But brain-brain interfaces——adding connections between brain tissues that normally aren't connected——might increase those abilities. These approaches use electrodes to read electrical signals from neurons, then transmit those signals (perhaps compressed/filtered/transformed) through wires / fiber optic cables / EM waves, then write them to other neurons through other electrodes. These are "copy nature's work" approaches, in the sense that we think nature made neurons that know how to arrange themselves usefully when connected with other neurons.
Problems with all electrical brain interface approaches
Massive cerebral prosthetic connectivity
Source: https://www.neuromedia.ca/white-matter/
Half of the human brain is white matter, i.e. neuronal axons with fatty sheaths around them to make them transmit signals faster. White matter is ~1/10 the volume of rodent brains, but ~1/2 the volume of human brains. Wiring is expensive and gets minimized; see "Principles of Neural Design" by Sterling and Laughlin. All these long-range axons are a huge metabolic expense. That means fast, long-range, high bandwidth (so to speak——there are many different points involved) communication is important to cognitive capabilities. See here.
A better-researched comparison would be helpful. But vaguely, my guess is that if we compare long-range neuronal axons to metal wires, fiber optic cables, or EM transmissions, we'd see (amortized over millions of connections): axons are in the same ballpark in terms of energy efficiency, but slower, lower bandwidth, and more voluminous. This leads to:
See "Prosthetic connectivity" for discussion of variants and problems. The main problem is that current brain implants furnish <10^4 connections, but >10^6 would probably be needed to have a major effect on problem-solving ability, and electrodes tend to kill neurons at the insertion site. I don't know how to accelerate this, assuming that Neuralink is already on the ball well enough.
Human / human interface
If one person could think with two brains, they'd be much smarter. Two people connected is not the same thing, but could get some of the benefits. The advantages of an electric interface over spoken language are higher bandwidth, lower latency, less cost (producing and decoding spoken words), and potentially more extrospective access (direct neural access to inexplicit neural events). But it's not clear that there should be much qualitative increase in philosophical problem-solving ability.
A key advantage over prosthetic connectivity is that the benefits might require a couple ooms fewer connections. That alone makes this method worth trying, as it will be probably be feasible soon.
Interface with brain tissue in a vat
The advantage of this approach is that it would in principle be scalable. The main additional obstacle, beyond any neural-neural interface approaches, is growing cognitively useful tissue in vitro. This is not completely out of the question——see "DishBrain"——but who knows if it would be feasible.
Massive neural transplantation
The approach
There have been some experiments along these lines, at a smaller scale, aimed at treating brain damage.
The idea is simply to scale up the brain's computing wetware.
Problems
Support for thinking
Generally, these approaches try to improve human thinking by modifying the algorithm-like elements involved in thinking. They are "figure it out ourselves" approaches.
The approaches
There is external support:
E.g. the printing press, the text editor, the search engine, the typechecker.
There is mental software:
E.g. the practice of mathematical proof, the practice of noticing rationalization, the practice of investigating boundaries.
There is social software:
E.g. a shared narrative in which such-and-such cognitive tasks are worth doing, the culture of a productive research group.
E.g. git.
Problems
FAQ
What about weak amplification
Getting rid of lead poisoning should absolutely be a priority. It won't greatly increase humanity's maximum intelligence level though.
What about ...
The real intelligence enhancement is ...
Look, I'm all for healing society, healing trauma, increasing collective consciousness, creating a shared vision of the future, ridding ourselves of malign egregores, blah blah. I'm all for it. But it's a difficult, thinky problem. ...So difficult that you might need some good thinking help with that thinky problem...
Is this good to do?
Yeah, probably. There are many downside risks, but the upside is large and the downsides can be greatly alleviated.