How to Make Superbabies

kman

One of the most important projects in the world. Somebody should fund it.

I think this project should receive more red-teaming before it gets funded.

Naively, it would seem that the "second species argument" matches much more strongly to the creation of a hypothetical Homo supersapiens than it does to AGI.

We've observed many warning shots regarding catastrophic human misalignment. The human alignment problem isn't easy. And "intelligence" seems to be a key part of the human alignment picture. Humans often lack respect or compassion for other animals that they deem intellectually inferior -- e.g. arguing that because those other animals lack cognitive capabilities we have, they shouldn't be considered morally relevant. There's a decent chance that Homo supersapiens would think along similar lines, and reiterate our species' grim history of mistreating those we consider our intellectual inferiors.

It feels like people are deferring to Eliezer a lot here, which seems unjustified given how much strategic influence Eliezer had before AI became a big thing, and how poorly things have gone (by Eliezer's own lights!) since then. There's been very little reasoning transparency in Eliezer's push for genetic enhancement. I just don't see why we're deferring to Eliezer so much as a strategist, when I struggle to name a single major strategic success of his.

[-]TsviBT8mo254

You shouldn't and won't be satisfied with this alone, as it doesn't deal with or even emphasize any particular peril; but to be clear, I have definitely thought about the perils: https://berkeleygenomics.org/articles/Potential_perils_of_germline_genomic_engineering.html

[-]Purplehermann8mo147

Right now only low-E tier human intelligences are being discussed, they'll be able to procreate with humans and be a minority.

Considering current human distributions and a lack of 160+ IQ people having written off sub-100 IQ populations as morally useless I doubt a new sub-population at 200+ is going to suddenly turn on humanity

If you go straight to 1000IQ or something sure, we might be like animals compared to them

-4Daphne_W6mo

100 IQ average humans have attempted to wipe out 60 IQ average humans. There were programs for mandatory or financially incentivized sterilization of people with down syndrome and other mental disabilities in many western countries, including their mass murder in the holocaust. As for >140 IQ average humans, remember that these make up little more than 3% of the population of every country. It would be very unstrategic for them to talk about the rest of humanity in an openly antagonistic way. Instead, if they wanted to win eugenics, they would have to make false promises to bring the rest of humanity along while they amass power. And this still begs the question that those with the money and power to do this would be trying to do eugenics in a way you consider to be right. That they wouldn't outgroup people or aspects of humanity that you care about. That the new subpopulation's memes would be aligned with the rest of humanity.

[-]Purplehermann6mo1512

Not sure why the holocaust is relevant.

60 IQ humans are defective, generally incapable of taking care of themselves, are unproductive and without a lot of effort by others have bad lives.

Of course we'd rather new humans be 100 IQ rather than 60 IQ. Even so, it's generally accepted that killing them, forcefully sterilizing (at least those who are aware) is immoral. Which means that despite incentives the majority population treats them as members of the same race and morally relevant.

(Contrast with current humanity - people are useful and can produce more than they eat, so there's no incentive to get rid of them. Further, you only need more selection for improvement, no sterilization necessary)

This new subpopulation would be much smaller than the 140+IQ group (I doubt we're getting 2 million 200IQ+ people very quickly), and also 140IQ people aren't interested in killing off everyone else...

If this new population wants to agitate for making new humans with high IQs I don't see that as a threat.

Eugenics for memes seems closer to sci-fi right now and unless done really stupidly seems irrelevant

-7Daphne_W6mo

2Mr Beastly8mo

Yes, and... "Would be interesting to see this research continue in animals. E.g. Provide evidence that they've made a "150 IQ" mouse or dog. What would a dog that's 50% smarter than the average dog behave like? or 500% smarter? Would a dog that's 10000% smarter than the average dog be able to learn, understand and "speak" in human languages?" -- From this comment

1Harry Lee-Jones6mo

May I offer an alternative view on intelligence and exploitation? I think universally, people exploit others to gain resources because of resource scarcity. As people become smarter, I concede they do become more capable of exploiting others for resources. However, smarter people are also more capable of generating resources by ethical means, and they are more aware of ethical considerations. I expect that increasing intelligence will decrease global resource scarcity, which will decrease people's desperation to exploit others for resources. Therefore, even though increased intelligence enables more exploitation, it will decrease the amount of exploitation we see by removing the incentive that is resource scarcity.

1Ebenezer Dukakis6mo

These points have merit, but they also work for intelligent AI.

-3NickH8mo

If your world view requires valuing the ethics of (current) people of lower IQ over those of (future) people of higher IQ then you have a much bigger problem than AI alignment. Whatever IQ is, it is strongly correlated with success which implies a genetic drive towards higher IQ, so your feared future is coming anyway (unless AI ends us first) and there is nothing we can logically do to have any long term influence on the ethics of smarter people coming after us.

[-]niplav8mo297

At the end of 2023, MIRI had ~$19.8 mio. in assets. I don't know much about the legal restrictions of how that money could be used, or what the state for financial assets is now, but if it's similar then MIRI could comfortably fund Velychko's primate experiments, and potentially some additional smaller projects.

(Potentially relevant: I entered the last GWWC donor lottery with the hopes of donating the resulting money to intelligence enhancement, but wasn't selected.)

5Mateusz Bagiński8mo

Possible that MIRI would like to avoid risking negative reputational consequences of supporting what is still as pretty anti-kosher in the mainstream.

9niplav8mo

Looked unlikely to me given the most-publicly-associated-with-MIRI person is openly & loudly advocating for funding this kind of work. But maybe the association isn't as strong as I think.

[-]kave8mo255

Copying over Eliezer's top 3 most important projects from a tweet:

1. Avert all creation of superintelligence in the near and medium term.
2. Augment adult human intelligence.
3. Build superbabies.

5Noosphere898mo

TBH, I don't particularly think it's one of the most important projects right now, due to several issues: 1. There's no reason to assume that we could motivate them any better than what we already do, unless we are in the business of changing personality, which carries it's own problems, or we are willing to use it on a massive scale, which simply cannot be done currently. 2. We are running out of time. The likely upper bound for AI that will automate basically everything is 15-20 years from Rafael Harth and Cole Wyeth, and unfortunately there's a real possibility that the powerful AI comes in 5-10 years, if we make plausible assumptions about scaling continuing to work, and given that there's no real way to transfer any breakthroughs to the somatic side of gene editing, it will be irrelevant by the time AI comes. Thus, human intelligence augmentation is quite poor from a reducing X-risk perspective.

8jessicata8mo

On EV grounds, "2/3 chance it's irrelevant because of AGI in the next 20 years" is not a huge contributor to the EV of this. Because, ok, maybe it reduces the EV by 3x compared to what it would otherwise have been. But there are much bigger than 3x factors that are relevant. Such as, probability of success, magnitude of success, cost effectiveness. Then you can take the overall cost effectiveness estimate (by combining various factors including probability it's irrelevant due to AGI being too soon) and compare it to other interventions. Here, you're not offering a specific alternative that is expected to pay off in worlds with AGI in the next 20 years. So it's unclear how "it might be irrelevant if AGI is in the next 20 years" is all that relevant as a consideration.

4Noosphere898mo

Usually, the other interventions I compare it to are preparing for AI automation of AI safety by doing preliminary work to control/align those AIs, or AI governance interventions that are hopefully stable for a very long time, and at least for the automation of AI safety, I assign much higher magnitudes of success, conditioning on success, like multiple OOMs combined with moderately better cost effectiveness and quite larger chances of success than the genetic engineering approach. To be clear, the key variable is conditional on success, the magnitude of that success is very, very high in a way that no other proposal really has, such that even with quite a lot lower probabilities for success than me, I'd still consider preparing for AI automation of AI safety and doing preliminary work such that we can trust/control these AIs to be the highest value alignment target by a mile.

3jessicata8mo

Oh, to be clear I do think that AI safery automation is a well targeted x risk effort conditioned on the AI timelines you are presenting. (Related to Paul Christiano alignment ideas, which are important conditional on prosaic AI)

3teradimich8mo

EY is known for considering humanity almost doomed. He may think that the idea of human intelligence augmentation is likely to fail. But it's the only hope. Of course, many will disagree with this. He writes more about it here or here.

3Noosphere898mo

The problem is that from a relative perspective, human augmentation is probably more doomed than AI safety automation, which in turn is more doomed than AI governance interventions, though I may have gotten the relative ordering of AI safety automation and I think the crux is I do not believe in the timeline for human genetic augmentation in adults being only 5 years, even given a well-funded effort, and I'd expect it to take 15-20 years, minimum for large increases in adult intelligence, which basically rules out the approach given the very likely timelines to advanced AI either killing us all or being aligned to someone.

6teradimich8mo

Yudkowsky may think that the plan 'Avert all creation of superintelligence in the near and medium term — augment human intelligence' has <5% chance of success, but your plan has <<1% chance. Obviously, you and he disagree not only on conclusions, but also on models.

4Purplehermann8mo

He already addressed this. If somehow international cooperation gives us a pause on going full AGI or at least no ASI - what then? Just hope it never happens, like nuke wars? The answer now is to set later generations up to be more able. This could mean doing fundamental research (whether in AI alignment or international game theory or something else), it could mean building institutions to enable it, and it could mean making them actually smarter. Genes might be the cheapest/easist way to affect marginal chances given the talent already involved in alignment and the amount of resources required to get involved politically or in building institutions

2Noosphere898mo

The answer is no, but this might have to happen under certain circumstances. The usual case (assuming that the government bans or restricts compute resources, and/or limits algorithimic research), is to use this time to either let the government fund AI alignment research, or go for a direct project to make AIs that are safe to automate AI safety research, and given that we don't have to race against other countries, we could afford far more safety taxes than usual to make AI safe. I think the key crux is I don't particularly think genetic editing is the cheapest/easiest way to affect marginal chances of doom, because of time lag plus needing to reorient the entire political system, which is not cheap, and the cheapest/easiest strategy to me to affect doom probabilities is to do preparatory AI alignment/control schemes such that we can safely hand off the bulk of the alignment work to the AIs, which then solve the alignment problem fully.

2Purplehermann8mo

Your direction sounds great - but how well can $4M move the needle there? How well can genesmith move the needle with his time and energy? I think you're correct about the cheapest/easist strategy in general, but completely off in regards to marginal advantages. Major labs will already be pouring massive amounts of money and human capital into direct AI alignment and using AIs to align AGI if we get to a freeze, and the further along in capabilities we get the more impactful such research would be. Genesmith's strategy benefits much more from starting now and has way less human talent and capital involved, hence higher marginal value

3Michael Harrop5mo

Have you discussed or tried petitioning the government to focus on this? I've been in D.C. for the past 8 months lobbying the government for something related. I've learned a lot about how the government works, and I've been thinking that it could be quite effective if people like you, Yuval Noah Harari, Max Tegmark, Geoffrey Hinton, etc., formed a small lobby group to get government action on this. It's very much related to the current "Make America Healthy Again" focus.

[-]David Gross8mo6321

How robust are these calculations against the possibility that individual gene effects aren't simply additional but might even not play well together? i.e. gene variant #1 raises your IQ by 2 points, variant #2 raises your IQ by 1 point, but variants #1+2 together make you able to multiply twelve-digit numbers in your head but unable to tie your shoes; or variant #3 lifts your life expectancy by making you less prone to autoimmune disease A, variant #4 makes you less prone to autoimmune disease B, but variants #3+4 together make you succumb to the common cold because your immune system is not up to the task.

It's hard for me to tell from the level of detail in your explanation here, but at times it seems like you're just naively stacking the ostensible effects of particular gene variants one on top of the other and then measuring the stack.

[-]GeneSmith8mo*6410

It's a good question. The remarkable thing about human genetics is that most of the variants ARE additive.

This sounds overly simplistic, like it couldn't possible work, but it's one of the most widely replicated results in the field.

There ARE some exceptions. Personality traits seem to be mostly the result of gene-gene interactions, which is one reason why SNP heritability (additive variance explained by common variants) is so low.

But for nearly all diseases and for many other traits like height and intelligence, ~~~80% of variance is additive~~. somewhere between 50 and 80% of the heritable variance is additive. And basically ALL of the variance we can identify with current genetic predictors is additive.

This might seem like a weird coincidence. After all, we know there is a lot of non-linearity in actual gene regulatory networks. So how could it be that all the common variants simply add together?

There's a pretty clear reason from an evolutionary point of view: evolution is able to operate on genes with additive effects much more easily than on those with non-additive effects.

The set of genetic variants inherited is scrambled every generation during the sperm and egg formation proces... (read more)

[-]jimrandomh8mo3514

The remarkable thing about human genetics is that most of the variants ARE additive.

I think this is likely incorrect, at least where intelligence-affecting SNPs stacked in large numbers are concerned.

To make an analogy to ML, the effect of a brain-affecting gene will be to push a hyperparameter in one direction or the other. If that hyperparameter is (on average) not perfectly tuned, then one of the variants will be an enhancement, since it leads to a hyperparameter-value that is (on average) closer to optimal.

If each hyperparameter is affected by many genes (or, almost-equivalently, if the number of genes greatly exceeds the number of hyperparameters), then intelligence-affecting traits will look additive so long as you only look at pairs, because most pairs you look at will not affect the same hyperparameter, and when they do affect the same hyperparameter the combined effect still won't be large enough to overshoot the optimum. However, if you stack many gene edits, and this model of genes mapping to hyperparameters is correct, then the most likely outcome is that you move each hyperparameter in the correct direction but overshooting the optimum. Phrased slightly differently: in... (read more)

9kman8mo

I definitely don't expect additivity holds out to like +20 SDs. We'd be aiming for more like +7 SDs.

2kave8mo

From population mean or from parent mean?

4GeneSmith8mo

Population mean

1idiotomic6mo

if you're applying the concept of +7 SDs seriously here (let alone +20 SDs) I'm almost certain you're grossly misusing the concept of a standard deviation. and since you're a coauthor of this post, it strongly suggests to me that the analysis done here is unreliable.

2GeneSmith6mo

Care to explain how you think it's being misused?

2idiotomic6mo

Standard deviations are used to characterize the spread around the mean of a normal distribution -- it is not intended to characterize the tails. This is why discussion around it tends to focus on the 1-2 SDs, where the bulk of the data is, and rarely 3-4 SDs -- it is rare to have the data (of sufficient size or low noise) to support meaningful interpretation of even 4 SDs with real-world data. So in practice, using precise figures like 5, 7, or 20 SDs is misleading, because the tails aren't usually sufficiently characterized (and it certainly isn't with intelligence) -- all you can really say is that it's beyond the validated range of the test. It's like taking seriously a measurement of 151.887 when the instrument operates in integers up to 10 -- you're implying you're meaningfully operating on a level of precision and range that you don't realistically have. It comes across as incredibly careless with regard to statistical nuance and rigor.

1Pablo Villalobos8mo

I suspect the analogy does not really work that well. Much of human genetic variation is just bad mutations that take a while to be selected out. For example, maybe a gene variant slightly decreases the efficiency of your neurons and makes everything in your brain slightly slower

4TsviBT8mo

Is this what you meant to say? Citation?

2kman8mo

This paper found that the heritability of most traits is ~entirely additive, supposedly including IQ according to whatever reference I followed to the paper, though I couldn't actually find where in the paper it said/implied that.

5TsviBT8mo

And then suddenly it's different for personality? Kinda weird.

6Steven Byrnes8mo

I have a simple model with toy examples of where non-additivity in personality and other domains comes from, see §4.3.3 here.

[-]TsviBT8mo102

Thanks. I think it's an important point you make; I do have it in mind that traits can have nonlinearities at different "stages", but I hadn't connected that to the personality trait issue. I don't immediately see a very strong+clear argument for personality traits being super exceptional here. Intuitively it makes sense that they're more "complicated" or "involve more volatile forces" or something due to being mental traits, but a bit more clarity would help. In particular, I don't see the argument being able to support yes significant broadsense heritability but very little apparent narrowsense heritability. (Though maybe it can somehow.)

(Also btw I wouldn't exactly call multiplicativity by itself "nonlinearity"! I would just say that after the genomic fan-in there is a nonlinearity. It's linearity as long as there's a latent variable that's a sum of alleles. Indeed, as E.G. pointed out to me, IQ could very well be like this, i.e. IQ-associated traits might be even better predicted by assuming IQ is lognormal (or some other such distribution). Though of course then you can't say that such and such downstream outcome is linear in genes; but the point would be that you can push alo... (read more)

4Steven Byrnes8mo

You’re not the first to complain about my terminology here, but nobody can tell me what terminology is right. So, my opinion is: “No, it’s the genetics experts who are wrong” :) If you take some stupid outcome like “a person’s fleep is their grip strength raised to the power of their alcohol tolerance”, and measure fleep across a population, you will obviously find that there’s a strong non-additive genetic contribution to that outcome. A.k.a. epistasis. If you want to say “no, that’s not really non-additive, and it’s not really epistasis, it’s just that ‘fleep’ is a damn stupid outcome to analyze”, then fine, but then the experts really need to settle on a standardized technical term for “damn stupid outcomes to analyze”, and then need to consider the possibility that pretty much every personality trait and mental health diagnosis (among other things) is a “damn stupid outcome” in much the same way. I do hope to unravel the deep structure of personality variation someday! In particular, what exactly are the linear “traits” that correspond directly to brain algorithm settings and hyperparameters? (See A Theory of Laughter and Neuroscience of human social instincts: a sketch for the very early stages of that. Warning: long.) I guess a generic answer would be: the path FROM brain algorithm settings and hyperparameters TO decisions and preferences passes through a set of large-scale randomly-initialized learning algorithms churning away for a billion seconds. (And personality traits are basically decisions and preferences—see examples here.) That’s just a massive source of complexity, obfuscating the relationship between inputs and outputs. A kind of analogy is: if you train a set of RL agents, each with slightly different reward functions, their eventual behavior will not be smoothly variant. Instead there will be lots of “phase shifts” and such. So again, we have “a set of large-scale randomly-initialized learning algorithms that run for a billion seconds” on th

[-]TsviBT8mo104

That’s just a massive source of complexity, obfuscating the relationship between inputs and outputs.

A kind of analogy is: if you train a set of RL agents, each with slightly different reward functions, their eventual behavior will not be smoothly variant. Instead there will be lots of “phase shifts” and such.

And yet it moves! Somehow it's heritable! Do you agree there is a tension between heritability and your claims?

4Steven Byrnes8mo

No … I think you must not have followed me, so I’ll spell it out in more detail. Let’s imagine that there’s a species of AlphaZero-chess agents, with a heritable and variable reward function but identical in every other way. One individual might have a reward function of “1 for checkmate”, but another might have “1 for checkmate plus 0.1 for each enemy piece on the right side of the endgame board” or “1 for checkmate minus 0.2 for each enemy pawn that you’ve captured”, or “0 for checkmate but 1 for capturing the enemy queen”, or whatever. Then every one of these individuals separately grows into “adults” by undergoing the “life experience” of the AlphaZero self-play training regime for 40,000,000 games. And now we look at the behavior of those “adults”. If you take two identical twin adults in this species, you’ll find that they behave extremely similarly. If one tends to draw out the enemy bishop in thus-and-such situation, then so does the other. Why? Because drawing out the enemy bishop is useful given a certain kind of reward function, and they have the same reward function, and they’re both quite good at maximizing it. So you would find high broad-sense heritability of behavior. But it’s unlikely that you’ll find a linear map from the space of reward functions to the space of midgame behavioral tendencies. Lots of individuals will be trying to draw out the enemy bishop in such-and-such situation, and lots of individuals won’t be trying to draw out the enemy bishop in that same kind of situation, for lots of different reasons, ultimately related to their different reward functions. The midgame behavior is more-or-less a deterministic function of the reward function, but it’s a highly nonlinear function. So you would measure almost zero narrow-sense heritability of behavior.

4TsviBT8mo

Wait a minute. Does your theory predict that heritability estimates about personality traits derived from MZTwins will be much much higher than estimates derived from DZTwins or other methods not involving MZTwins?

4Steven Byrnes8mo

Yeah, one of the tell-tale signs of non-additive genetic influences is that MZ twins are still extremely similar, but DZ twins and more distant relatives are more different than you’d otherwise expect. (This connects to PGSs because PGSs are derived from distantly-related people.) See §1.5.5 here, and also §4.4 (including the collapsible box) for some examples.

4TsviBT8mo

Mkay.... I'm gonna tap out for now, but this is very helpful, thanks. I'm still pretty skeptical, though indeed 1. I am failing (and then I think later succeeding) at significant chunks of basic reading comprehension about what you're saying; 2. I'm still confused, so my skepticism isn't a confident No. As a bookmark/trailhead, I suggest that maybe your theory of "personality has a high complexity but pretty deterministic map from a smallish number of pretty-genetically-linear full-brain-settings to behavior due to convergent instrumentality" and some sort of "personality mysteriously has a bunch of k-th order epistases that all add up" would both predict MZ being more similar than DZ, but your theory would predict this effect more strongly than the k-th order thing. Another: there's something weird where I don't feel your argument about a complex map being deterministic because of convergent instrumentality ought to work for the sorts of things that personality traits are; like they don't seem analogous to "draws out bishop in xyz position", and in the chess example idk if I especially would there to be "personality traits" of play.... or something about this.

2TsviBT8mo

Another bookmark: IIUC your theory requires that the relevant underlying brain factors are extremeley pinned down by genetics, because the complicated map from underlying brain stuff to personality is chaotic.

8TsviBT8mo

Hm. I wrote half a response to this, but then realized that... IDK, we're thinking of this incorrectly, but I'm not fully sure how. (Ok possibly only I'm thinking of it incorrectly lol. Though I think you are too.) I'll say some things, but not sure what the overall point should be. (And I still haven't read your post so maybe you addressed some of this elsewhere.) (And any of the following might be confused, I'm not being careful.) Thing: You could have a large number of genetic variants Gk in a genome G, and then you have a measured personality trait p(G). Suppose that in fact p(G)=f(∑kwkGk). If f is linear, like f(x)=ax+b, then p is unproblematically linear. But suppose f is exponentiation, so that p(G)=e∑kwkGk=∏kwkGk In this case, p is of course not linear in the Gk. However: * This does not make p a stupid thing to measure in any way. If it's an interesting trait then it's an interesting trait. * You could absolutely still affect p straightforwardly using genomic engineering; just up/downvote Gk with positive/negative wk. * There's an obvious latent, ∑kwkGk, which is linear. This latent should be of interest; and this doesn't cut against p being real, or vice versa. * In real life, you get data that's concentrated around one value of ∑kwkGk, with gaussian variation. If you look at ∑k≠iwkGk, it's also concentrated. So like, take the derivative of ewig+∑k≠iwkGk with respect to g. (Or something like that.) This should tell us something like the "additive" effect of Gi! In other words, around the real life mean, it should be locally kinda linear. * This implies you should see the "additive" variance! Your GWAS should pick stuff up! * So if your GWAS is not picking stuff up, then p(G) is NOT like this! Thing: A simple alternative definition of epistasis (which I don't actually like, but is better than "nonlinear"): There are epistatic effects on p(G) from G when p(G) cannot be written in the form p(G)=f(∑kwkGk) with f:R→R arbitrary. Thing: Suppose n

3Steven Byrnes8mo

Even in personality and mental health, the PGSs rarely-if-ever account for literally zero percent of the variance. Normally the linear term of a Taylor series is not zero. I think what you’re missing is: the linear approximation only works well (accounts for much of the variance) to the extent that the variation is small. But human behavioral differences—i.e. the kinds of things measured by personality tests and DSM checklists—are not small. There are people with 10× more friends than me, talk to them 10× as often as me, play 10× more sports than me, read 10× less than me, etc. Why? As in my other comment, small differences in what feels rewarding and motivating to a person can cascade into massive differences in behavior. If someone finds it mildly unpleasant to be around other people, then that’s a durable personality trait, and it impacts a million decisions that they’ll make every day, all in the same direction, and thus it impacts how they spend pretty much all of their waking time, including even what they choose to think about each moment, for their entire life. So much flows from that.

2tailcalled8mo

It becomes more complex once you take the sum of the product of several things. At that point the log-additive effect of one of the terms in the sum disappears if the other term in the sum is high. If you've got a lot of terms in the sum and the distribution of the variables is correct, this can basically kill the bulk of common additive variance. Conceptually speaking, this can be thought of as "your system is a mixture of a bunch of qualitatively distinct things". Like if you imagine divorce or depression can be caused by a bunch of qualitatively unrelated things.

2TsviBT8mo

Hm.... Not sure how to parse this. (What do you mean " the distribution of the variables is correct"?) Isn't the derivative of the full variable in one of the multiplicands still noticeable? Maybe it would help if you make some quantitative statement? I mean, I think depression is heritable, and I think there are polygenic scores that do predict some chunk of this. (From a random google: https://jamanetwork.com/journals/jamapsychiatry/fullarticle/2783096 ) Quite plausibly yes these heritability estimates and PGSes are picking up on heterogeneous things, but they still work, and you can still construct the PGS; you find the additive variants when you look. (Also I am interested in the difference between traits that are OR / SUM of some heritable things and some non-heritable things. E.g. you can get lung cancer from lung cancer genes, or from smoke 5 packs a day. This matters for asking "just how low exactly can we drive down disease risk?". But this would not show up as missing heritability!)

2tailcalled8mo

Taking the logarithm (to linearize the association) scales the derivative down by the reciprocal of the magnitude. So if one of the terms in the sum is really big, all the derivatives get scaled down by a lot. If each of the terms are a product, then the derivative for the big term gets scaled up to cancel out the downscaling, but the small terms do not. Under the condition I mentioned, polygenic scores will tend to focus on the traits that cause the most common kind of depression, while neglecting other kinds. The missing heritability will be due to missing those other kinds.

2TsviBT8mo

Can you please write down the expressions you're talking about as math? If you're trying to invoke standard genetics knowledge, I'm not a geneticist and I'm not picking it up from what you're saying.

2tailcalled7mo

Let's start with the basics: If the outcome f is a linear function of the genes x, that is f(x)=βx, then the effect of each gene is given by the gradient of f, i.e. ∇xf(x)=β. (This is technically a bit sketchy since a genetic variant is discrete while gradients require continuity, but it works well enough as a conceptual approximation for our purposes.) Under this circumstance, we can think of genomic studies as finding β. (This is also technically a bit sketchy because of linkage disequillibrium and such, but it works well enough as a conceptual approximation for our purposes.) If f isn't a linear function, then there is no constant β to find. However, the argument for genomic studies still mostly goes through that they can find E[∇xf(x)], it's just that this expression now denotes a weird mismash effect size that's not very interpretable. As you observed, if f is almost-linear, for example if f(x)=eβx, then genomic studies still have good options. The best is probably to measure the genetic influence on logf, as then we get a pretty meaningful coefficient out of it. (If we measured the genetic influence of f without the logarithm, I think under commonly viable assumptions we would get β′i∝eβi−1, but don't cite me on that.) The trouble arises when you have deeply nonlinear forms such as f(x)=eβx+eγx. If we take the gradient of this, then the chain rule gives us ∇logf(x)=eβxβ+eγxγeβx+eγx. That is, the two different mechanisms "suppress" each other, so if eβx is usually high, then the γ term would usually be (implicitly!) excluded from the analysis.

2TsviBT7mo

Ah. Thank you, this makes sense of what you said earlier. (I / someone could have gotten this from what you had written before, by thinking about it more, probably.) I agree with your analysis as math. However, I'm skeptical of the application to the genetics stuff, or at least I don't see it yet. Specifically, you wrote: And your argument here says that there's "gradient interference" between the summed products specifically when one of the summed products is really big. But in the case of disease risk, IIUC the sum-of-products f(x) is something like logits. So translating your argument, it's like: In this case, yes the analysis is valid, but it's not very relevant. For the diseases that people tend to talk about, if there are several substantial disjunctive causes (I mean, the risk is a sum of a few different sub-risks), then they all would show substantial signal in the data. None of them drowns out all the others. Maybe you just meant to say "In theory this could happen". Or am I missing what you're suggesting? E.g. is there a way for there to be a trait that: * has lots of variation (e.g. lots of sick people and lots of non-sick people), and * it's genetic, and * it's a fairly simple functional form like we've been discussing, * but you can't optimize it much by changing a bunch of variants found by looking at some millions of genotype/phenotype pairs?

2tailcalled7mo

The original discussion was about how personality traits and social outcomes could behave fundamentally differently from biological traits when it comes to genetics. So this isn't necessarily meant to apply to disease risks.

2TsviBT7mo

Well you brought up depression. But anyway, all my questions apply to personality traits as well. ..... To rephrase / explain how confused I am about what you're trying to tell me: It kinda sounds like you're saying "If some trait is strongly determined by one big chunk of genes, then you won't be able to see how some other chunk affects the trait.". But this can't explain missing heritability! In this scenario, none of the heritability is even from the second chunk of genes in the first place! Or am I missing something?

2tailcalled7mo

Some of the heritability would be from the second chunk of genes.

2TsviBT7mo

To the extent that the heritability is from the second chunk, to that extent the gradient does flow, no?

2tailcalled7mo

Why?

2TsviBT7mo

Because if some of the heritability is from the second chunk, that means that for some pairs of people, they have roughly the same first chunk but somewhat different second chunks; and they have different traits, due to the difference in second chunks. If some amount of heritability is from the second chunk, then to that extent, there's a bunch of pairs of people whose trait differences are explained by second chunk differences. If you made a PGS, you'd see these pairs of people and then you'd find out how specifically the second chunk affects the trait. I could be confused about some really basic math here, but yeah, I don't see it. Your example for how the gradient doesn't flow seems to say "the gradient doesn't flow because the second chunk doesn't actually affect the trait".

2tailcalled7mo

This only applies if the people are low in the first chunk and differ in the second chunk. Among the people who are high in the first chunk but differ in the second chunk, the logarithm of their trait level will be basically the same regardless of the second chunk (because the logarithm suppresses things by the total), so these people will reduce the PGS coefficients rather than increasing the PGS coefficients. When you create the PGS, you include both groups, so the PGS coefficients will be downwards biased relative to γ.

2TsviBT7mo

Wouldn't this also decrease the heritability?

2tailcalled7mo

It would decrease the narrowsense (or additive) heritability, which you can basically think of as the squared length of your coefficient vector, but it wouldn't decrease the broadsense heritability, which is basically the phenotypic variance in expected trait levels you'd get by shuffling around the genotypes. The missing heritability problem is that when we measure these two heritabilities, the former heritability is lower than the latter.

2TsviBT7mo

Why not? Shuffling around the second chunk, while the first chunk is already high, doesn't do anything, and therefore does not contribute phenotypic variance to broadsense heritability.

[-]tailcalled7mo100

Ok, more specifically, the decrease in the narrowsense heritability gets "double-counted" (after you've computed the reduced coefficients, those coefficients also get applied to those who are low in the first chunk and not just those who are high, when you start making predictions), whereas the decrease in the broadsense heritability is only single-counted. Since the single-counting represents a genuine reduction while the double-counting represents a bias, it only really makes sense to think of the double-counting as pathological.

2TsviBT7mo

Ah... ok I think I see where that's going. Thanks! (Presumably there exists some standard text about this that one can just link to lol.) I'm still curious whether this actually happens.... I guess you can have the "propensity" be near its ceiling.... (I thought that didn't make sense, but I guess you sometimes have the probability of disease for a near-ceiling propensity be some number like 20% rather than 100%?) I guess intuitively it seems a bit weird for a disease to have disjunctive causes like this, but then be able to max out at the risk at 20% with just one of the disjunctive causes? IDK. Likewise personality...

4tailcalled7mo

I don't think so. For something like divorce, you could imagine the following causes: * Most common cause is you married someone who just sucks * ... but maybe you married a closeted gay person * ... or maybe your partner was good but then got cancer and you decided to abandon them rather than support them through the treatment The genetic propensities for these three things are probably pretty different: If you've married someone who just sucks, then a counterfactually higher genetic propensity to marry people who suck might counterfactually lead to having married someone who sucks more, but a counterfactually higher genetic propensity to marry a closeted gay person probably wouldn't lead to counterfactually having married someone who sucks more, nor have much counterfactual effect on them being gay (because it's probably a nonlinear thing), so only the genetic propensity to marry someone who sucks matters. In fact, probably the genetic propensity to marry someone who sucks is inversely related to the genetic propensity to divorce someone who encounters hardship, so the final cause of divorce is probably even more distinct from the first one.

2TsviBT7mo

How confident are you / why do you think this? (It seems fairly plausible given what I've heard about the field of genomics, but still curious.) E.g. "I have a genomics PhD" or "I talk to geneticists and they don't really know about this stuff" or "I follow some twitter stuff and haven't heard anyone talk about this". Ok I'm too tired to follow this so I'll tap out of the thread for now. Thanks again!

6tailcalled7mo

I talk to geneticists (mostly on Twitter, or rather now BlueSky) and they don't really know about this stuff.

2tailcalled8mo

Not right now, I'm on my phone. Though also it's not standard genetics math.

2TsviBT8mo

Ok. I don't get why you think this. It doesn't seem to make any sense. You'd still notice the effect of variants that cause depression-rare, exactly like if depression-rare was the only kind of depression. How is your ability to detect depression-rare affected by the fact that there's some genetic depression-common? Depression-common could just as well have been environmentally caused. I might be being dumb, I just don't get what you're saying and don't have a firm grounding myself.

2tailcalled8mo

It doesn't matter if depression-common is genetic or environmental. Depression-common leads to the genetic difference between your cases and controls to be small along the latent trait axis that causes depression-rare. So the effect gets estimated to be not-that-high. The exact details of how it fails depends on the mathematical method used to estimate the effect.

2TsviBT8mo

Ok I think I get what you're trying to communicate, and it seems true, but I don't think it's very relevant to the missing heritability thing. The situation you're describing applies to the fully linear case too. You're just saying that if a trait is more polygenic / has more causes with smaller effects, it's harder to detect relevant causes. Unless I still don't get what you're saying.

2tailcalled8mo

It kind-of applies to the Bernoulli-sigmoid-linear case that would usually be applied to binary diagnoses (but only because of sample size issues and because they usually perform the regression one variable at a time to reduce computational difficulty), but it doesn't apply as strongly as it does to the polynomial case, and it doesn't apply to the purely linear (or exponential-linear) case at all. If you have a purely linear case, then the expected slope of a genetic variant onto an outcome of interest is proportional to the effect of the genetic variant. The issue is in the polynomial case, the effect size of one genetic variant depends on the status of other genetic variants within the same term in the sum. Statistics gives you a sort of average effect size, but that average effect size is only going to be accurate for the people with the most common kind of depression.

1Archimedes8mo

I don’t find that surprising at all. IMO, personality is a more of an emergent balancing of multidimensional characteristics than something like height or IQ (though this is mostly vibes-based speculation).

1Roger Scott8mo

Does it seem likely that a trait that has survival significance (in a highly social animal such as a human) would be emergent? Even if it might have been initially, you'd think selective pressure would have brought forth a set of genes that have significant influence on it.

2Roger Scott8mo

I think the "second" question about second order effects was really the main question here. If the intentional beneficial effects don't quite add, that's no great tragedy, but if combining multiple edits produces unexpected changes, some of which are bad, that's kind of a deal killer. I don't find the chart you reference to be very convincing, since it only lists a handful of characteristics that are sufficiently common to have names and have been studied in such an analysis. For every one of those there are likely countless less frequent and/or more subtle "bad" variations whose correlation with the things we're trying to fix we have no idea of. Informally, don't you have to wonder why, if a small number of edits would seem to lead to a clearly superior genome, natural selection hasn't happened upon some of those combinations already? How can we know what combinations were "tried" earlier in our evolution with long-term negative consequences?

2p.b.8mo

Do you have a reference for the personality trait gene-gene interaction thing? Or maybe an explanation how that was determined?

1lumire8mo

If the variance for intelligence is primarily additive, then why are IQ GWAS heritability estimates significantly under the heritability estimates you see from twin studies (or even GWAS heritability for height)?

7kman8mo

The SNP heritability estimates for IQ of (h^2 = ~0.2) are primarily based on a low quality test that has a test-retest reliability of 0.6, compared to ~0.9 for a gold-standard IQ test. So a simple calculation to adjust for this gets you a predicted SNP heritability of 0.2 * (0.9 / 0.6)^2 = 0.45 0.2 * (0.9 / 0.6) = 0.30 for a gold standard IQ test, which matches the SNP heritability of height. As for the rest of the missing heritability: variants with frequency less than 1% aren't accounted for by the SNP heritability estimate, and they might contribute a decent bit if there are lots of them and their effects sizes are larger. EDIT: the original adjustment for test-retest reliability was incorrect: the correlations shouldn't be squared.

1Crazy philosopher8mo

An other problem with authors calculs of potential to improve intelligence: let's suppose, there is a problem in the human brain that reduces IQ by 10 points, and it can be solved by Gene1 or Gene2. Let's suppose that 99% of humans do not have either Gene1 or Gene2. In this case, the author's method would show that if we added both Gene1 and Gene2 to the same person, their IQ would increase by 20 points.

3GeneSmith8mo

I don't understand your question

4ChristianKl8mo

A lot of curves are sigmoid. Let's say there's a neurotransmitter where having to double the amount of it increases IQ but there are no gains from having four times as much of the neurotransmitter. There are two genes that both double the production of the neurotransmitter. If both genes individually are +5 IQ both genes together don't give you +10 IQ. It would even be possible that overproduction of that neurotransmitter produces problems at 4x the normal rate but not a 2x the normal rate. When it comes to chicken and their size I would expect the relationship of there being two genes that both increase muscle production to be happen more frequently than for intelligence. If you have genetic mutations that increase intelligence without cost evolution works to spread them through the whole population. If you have wild chicken for whom a given size is optimal there's no strong selection pressure to get rid of all the +x or -y size genes from the gene pool. One way to look into this would be to see how many of the genes that increase physical size more when there are two copies of the gene compared to how many genes increase intelligence more when there are two copies of it. And how many genes increase size/intelligence with one copy but decrease it with two copies.

[-]Kris Moore8mo585

The "Black Death selection" finding you mention was subject to a very strong rebuttal preprinted in March 2023 and published yesterday in Nature. The original paper committed some pretty basic methodological errors^[1] and, in my opinion, it's disappointing that Nature did not decide to retract it. None of their claims of selection – neither the headline ERAP2 variant or the "half a dozen others" you refer to – survive the rebuttal's more rigorous reanalysis. I do some work in ancient DNA and am aware of analyses on other datasets (published and unpublished) that fail to replicate the original paper's findings.

^{^}
Some of the most glaring (but not necessarily most consequential): a failure to correctly estimate the allele frequencies underlying the selection analysis; use of a genotyping pipeline poorly suited to ancient DNA which meant that 80% of the genetic variants they "analysed" were likely completely artefactual and did not exist.

[-]lemonhope8mo120

use of a genotyping pipeline poorly suited to ancient DNA which meant that 80% of the genetic variants they "analysed" were likely completely artefactual and did not exist.

Brutal!! I didn't know this gotcha existed. I hope there aren't too many papers silently gotch'd by it. Sounds like the type of error that could easily be widespread and unnoticed, if the statistical trace it leaves isn't always obvious.

3Kris Moore8mo

I don't think that recent ancient DNA papers are affected by this issue, at least not to the same extent. Every aDNA researcher I know is extremely aware of the many pitfalls associated with sequencing ancient material and the various chemical and computational methods to mitigate them. Checking for signs of systematic artifacts in your aDNA data is very routine and not especially difficult. To provide some brief speculation, I think a major explanation for this paper's errors is that aDNA lab that did the sequencing was quite old, under-staffed, and did not have much recent experience with sequencing human nuclear aDNA, so they had not kept fully abreast of the enormous methodological improvements in this area over the past twenty years.

[-]GeneSmith8mo111

Thanks for catching that! I hadn't heard. I will probably have to rewrite that section of the post.

What's your impression about the general finding about many autoimmune variants increasing protection against ancient plauges?

3Kris Moore8mo

Basically any paper trying to detect signals of natural selection in humans will turn up something plausibly immune-related [1] [2] [3] [4]. Alongside pigmentation and diet-related genes, it's one of the most robustly detected categories of monogenic selection signal. While it seems extremely likely that some selection due to pathogenic disease has occurred in humans, I don't think I've seen a paper that convincingly ties a particular selected gene to a particular historical pathogen or pandemic. It would be pretty hard to do so. There's a many-to-many mapping between immune system genes and pathogenic diseases, and selection generally takes many centuries to detectably alter allele frequencies, during which time there have generally been many epidemics and other changes to the environment – which is responsible for the selection signal? Regarding autoimmunity in particular: I am in no way an immunologist and have much less insight to offer here, but perhaps it isn't surprising (almost tautological?) that immune system genes are often implicated in autoimmunity as well. And I'm not sure that inborn immunity due to HLA alleles or similar will be an important tool in the human race's survival in the face of future pandemics. It's perhaps telling that when you try to find variants associated with getting critically ill with COVID-19 – an extremely well-powered examination of the effects of genetic variability on response to a pandemic disease! – the very largest effect sizes for individual variants are a doubling/halving of risk. This is a notable difference, but nowhere close to "total immunity" vs "certain death". Having said that, I view "robust pathogen defence vs autoimmunity trade-off" as a very plausible just-so story, and likely to be true, but not concretely established at present. Sadly, it's one of those questions in science where running the right controlled experiment is practically impossible and we have to make do with detective work.

[-]RichardJActon8mo2912

There are a couple of major problems with naively intervening to edit sites associated with some phenotype in a GWAS or polygenic risk score.

The SNP itself is (usually) not causal Genotyping arrays select SNPs the genotype of which is correlated with a region around the SNP, they are said to be in linkage with this region as this region tends to be inherited together when recombination happens in meiosis. This is a matter of degree and linkage scores allow thresholds to be set for how indicative a SNP is about the genotype a given region.
If it is not the SNP but rather something with which the SNP is in linkage that is causing the effect editing the SNP has no reason the effect the trait in question.
It is not trivial to figure out what in linkage with a SNP might be causing an effect.
Mendelian randomisation (explainer: https://www.bmj.com/content/362/bmj.k601) is a method that permits the identification of causal relationships from observational genetic studies which can help to overcome this issue.
In practice epistatic interactions between QTLs matter for effects sizes and you cannot naively add up the effect sizes of all the QTLs for a trait and expect the result to refl

... (read more)

4kman8mo

This is taken into account by our models, and is why we see such large gains in editing power from increasing data set sizes: we're better able to find the causal SNPs. Our editing strategy assumes that we're largely hitting non-causal SNPs. I'm not aware of any evidence for substantial effects of this sort on quantitative traits such as height. We're also adding up expected effects, and as long as those estimates are unbiased the errors should cancel out as you do enough edits. One thing we're worried about is cases where the haplotypes have the small additive effects rather than individual SNPs, and you get an unpredictable (potentially deleterious) effect if you edit to a rare haplotype even if all SNPs involved are common. Are you aware of any evidence suggesting this would be a problem?

8RichardJActon8mo

Could you expand on what sense you have 'taken this into account' in your models? What are you expecting to achieve by editing non-causal SNPs? The first paper I linked is about epistasic effects on the additivity of a QTLs for quantitative trait, specifically heading date in rice, so this is evidence for this sort of effect on such a trait. The general problem is without a robust causal understanding of what an edit does it is very hard to predict what shorts of problem might arise from novel combinations of variants in a haplotype. That's just the nature of complex systems, a single incorrect base in the wrong place may have no effect or cause a critical cascading failure. You don't know until you test it or have characterized the system so well you can graph out exactly what is going to happen. Just testing it in humans and seeing what happens is eventually going to hit something detrimental. When you are trying to do enhancement you tend to need a positive expectation that it will be safe not just no reason to think it won't be. Many healthy people would be averse to risking good health for their kid, even at low probability of a bad outcome.

5kman8mo

If we have a SNP that we're 30% sure is causal, we expect to get 30% of its effect conditional on it being causal. Modulo any weird interaction stuff from rare haplotypes, which is a potential concern with this approach. I didn't read your first comment carefully enough; I'll take a look at this.

4TsviBT8mo

Can you comment your current thoughts on rare haplotypes?

4kman8mo

Don't have much to say on it right now, I really need to do a deep dive into this at some point.

3RichardJActon8mo

I'm curious about the basis on which you are assigning a probability of causality without a method like mendelian randomisation, or something that tries to assign a probability of an effect based on interpreting the biology like a coding of the output of something like SnpEff to an approximate probability of effect. The logic of 30% of its effect based on 30% chance it's causal only seems like it will be pretty high variance and only work out over a pretty large number of edits. It is also assuming no unexpected effects of the edits to SNPs that are non-causal for whatever trait you are targeting but might do something else when edited.

5kman8mo

Using finemapping. I.e. assuming a model where nonzero additive effects are sparsely distributed among SNPs, you can do Bayesian math to infer how probable each SNP is to have a nonzero effect and its expected effect size conditional on observed GWAS results. Things like SnpEff can further help by giving you a better prior.

3RichardJActon8mo

(For people reading this thread who want an intro to finemapping this lecture is a great place to start for a high level overview https://www.youtube.com/watch?v=pglYf7wocSI)

3hold_my_fish6mo

This is a point of uncertainty that bothered me when I was doing a similar analysis a while ago. GWAS data is possibly good enough to estimate causal effects of haplotypes, but that's not enough information to do single base edits. To have reasonable confidence of getting the predicted effect, it'd be necessary to to make all the edits to transform the original haplotype into a different haplotype. And unlike with distant variants where additive effects dominate, it'd make sense if non-additive effects are strong locally, since the variants are near each other. Whether this is actually true in reality is way beyond my knowledge, though.

2lemonhope8mo

To dumb it down a bit, here's my made up example: you get +1 IQ if your brain has surplus oxygen in the blood flowing through it. There's 1000 ways to get a bit more oxygen in there, but with +1000 oxygen, you still only get +1 IQ. Is that the idea?

3RichardJActon8mo

Kind of, there are many ways that changed in isolation get you a bit more oxygen but many of them act through the same mechanism so you change 1000 things that get you +1 oxygen on their own but in combination only get you +500. To use a software analogy imagine an object with two methods where if you call either of them a property of an object is set to true, it doesn't matter if you call both methods or if you have a bunch of functions that call those methods you still just get true. Calling either method or any function that calls them is going to be slightly correlated with an increased probability the the property of the object will be true but it does not add. There are many way to make it true but making it true more times does not make it 'more true'. If we change this from a boolean to an integer then some methods might only increment it if it is not already greater than some value specific to the method.

[-]LWLW8mo2613

How much do people know about the genetic components of personality traits like empathy? Editing personality traits might be almost as or even more controversial than modifying “vanity” traits. But in the sane world you sketched out this could essentially be a very trivial and simple first step of alignment. “We are about to introduce agents more capable than any humans except for extreme outliers: let’s make them nice.” Also, curing personality disorders like NPD and BPD would do a lot of good for subjective wellbeing.

I guess I’m just thinking of a failure mode where we create superbabies who solve task-alignment and then control the world. The people running the world might be smarter than the current candidates for god-emperor, but we’re still in a god-emperor world. This also seems like the part of the plan most likely to fail. The people who would pursue making their children superbabies might be disinclined towards making their children more caring.

[-]GeneSmith8mo183

Very little at the moment. Unlike intelligence and health, a lot of the variance in personality traits seems to be the result of combinations of genes rather than purely additive effects.

This is one of the few areas where AI could potentially make a big difference. You need more complex models to figure out the relationship between genes and personality.

But the actual limiting factor right now is not model complexity, but rather data. Even if you have more complex models, I don't think you're going to be able to actually train them until you have a lot more data. Probably a minimum of a few million samples.

We'd like to look into this problem at some point and make scaling law graphs like the ones we made for intelligence and disease risk but haven't had the time yet.

[-]Ebenezer Dukakis8mo1914

This is starting to sound a lot like AI actually. There's a "capabilities problem" which is easy, an "alignment problem" which is hard, and people are charging ahead to work on capabilities while saying "gee, we'd really like to look into alignment at some point".

[-]TsviBT8mo1711

It's utterly different.

Humans are very far from fooming.
- Fixed skull size; no in silico simulator.
- Highly dependent on childhood care.
- Highly dependent on culturally transmitted info, including in-person.
Humans, genomically engineered or not, come with all the stuff that makes humans human. Fear, love, care, empathy, guilt, language, etc. (It should be banned, though, to remove any human universals, though defining that seems tricky.) So new humans are close to us in values-space, and come with the sort of corrigibility that humans have, which is, you know, not a guarantee of safety, but still some degree of (okay I'm going to say something that will trigger your buzzword detector but I think it's a fairly precise description of something clearly real) radical openness to co-creating shared values.

[-]Ebenezer Dukakis8mo117

Humans are very far from fooming.

Tell that to all the other species that went extinct as a result of our activity on this planet?

I think it's possible that the first superbaby will be aligned, same way it's possible that the first AGI will be aligned. But it's far from a sure thing. It's true that the alignment problem is considerably different in character for humans vs AIs. Yet even in this particular community, it's far from solved -- consider Brent Dill, Ziz, Sam Bankman-Fried, etc.

Not to mention all of history's great villains, many of whom believed themselves to be superior to the people they afflicted. If we use genetic engineering to create humans which are actually, massively, undeniably superior to everyone else, surely that particular problem is only gonna get worse. If this enhancement technology is going to be widespread, we should be using the history of human activity on this planet as a prior. Especially the history of human behavior towards genetically distinct populations with overwhelming technological inferiority. And it's not pretty.

So yeah, there are many concrete details which differ between these two situations. But in terms of high-level strategic implications, I think there are important similarities. Given the benefit of hindsight, what should MIRI have done about AI back in 2005? Perhaps that's what we should be doing about superbabies now.

[-]TsviBT8mo1715

Tell that to all the other species that went extinct as a result of our activity on this planet?

Individual humans.

Brent Dill, Ziz, Sam Bankman-Fried, etc.

These are incredibly small peanuts compared to AGI omnicide.
You're somehow leaving out all the people who are smarter than those people, and who were great for the people around them and humanity? You've got like 99% actually alignment or something, and you're like "But there's some chance it'll go somewhat bad!"... Which, yes, we should think about this, and prepare and plan and prevent, but it's just a totally totally different calculus from AGI.

5Noosphere898mo

I'd flag here that the 99% number seems very easy to falsify, solely based on the 20th century experience of both the 2 great wars, as well as the genocides/civil wars of the 20th century, and it's quite often that one human group is vastly unaligned to another human group, causing mass strife and chaos.

[-]TsviBT8mo131

I'm saying that (waves hands vigorously) 99% of people are beneficent or "neutral" (like, maybe not helpful / generous / proactively kind, but not actively harmful, even given the choice) in both intention and in action. That type of neutral already counts as in a totally different league of being aligned compared to AGI.

one human group is vastly unaligned to another human group

Ok, yes, conflict between large groups is something to be worried about, though I don't much see the connection with germline engineering. I thought we were talking about, like, some liberal/techie/weirdo people have some really really smart kids, and then those kids are somehow a threat to the future of humanity that's comparable to a fast unbounded recursive self-improvement AGI foom.

[-]Noosphere898mo127

I'm saying that (waves hands vigorously) 99% of people are beneficent or "neutral" (like, maybe not helpful / generous / proactively kind, but not actively harmful, even given the choice) in both intention and in action. That type of neutral already counts as in a totally different league of being aligned compared to AGI.

I think this is ultimately the crux, at least relative to my values, I'd expect at least 20% in America to support active efforts to harm me or my allies/people I'm altruistic to, and do so fairly gleefully (an underrated example here is voting for people that will bring mass harm to groups they hate, and hope that certain groups go extinct).

Ok, yes, conflict between large groups is something to be worried about, though I don't much see the connection with germline engineering. I thought we were talking about, like, some liberal/techie/weirdo people have some really really smart kids, and then those kids are somehow a threat to the future of humanity that's comparable to a fast unbounded recursive self-improvement AGI foom.

Okay, the connection was to point out that lots of humans are not in fact aligned with each other, and I don't particularly think superbabies are a threat to the future of humanity that is comparable to AGI, so my point was more so that the alignment problem is not naturally solved in human-to human interactions.

9TsviBT8mo

Ok... so I think I understand and agree with you here. (Though plausibly we'd still have significant disagreement; e.g. I think it would be feasible to bring even Hitler back and firmly away from the death fever if he spent, IDK, a few years or something with a very skilled listener / psychic helper.) The issue in this discourse, to me, is comparing this with AGI misalignment. It's conceptually related in some interesting ways, but in practical terms they're just extremely quantitatively different. And, naturally, I care about this specific non-comparability being clear because it says whether to do human intelligence enhancement; and in fact many people cite this as a reason to not do human IE.

5Noosphere898mo

Re human vs AGI misalignment, I'd say this is true, in that human misalignments don't threaten the human species, or even billions of people, whereas AI does, so in that regard I admit human misalignment is less impactful than AGI misalignment. Of course, if we succeed at creating aligned AI, than human misalignments matter much, much more. (Rest of the comment is a fun tangentially connected scenario, but ultimately is a hypothetical that doesn't matter that much for AI alignment.) At the very least, that would require him to not be in control of Germany by that point, and IMO most value change histories rely on changing their values in the child-teen years, because that's when their sensitivity to data is maximal. After that, the plasticity/sensitivity of values goes way down when you are an adult, and changing values is much, much harder.

4TsviBT8mo

Right, ok, agreed. I agree qualitatively, but I do mean to say he's in charge of Germany, but somehow has hours of free time every day to spend with the whisperer. If it's in childhood I would guess you could do it with a lot less contact, though not sure. TBC, the whisperer here would be considered a world-class, like, therapist or coach or something, so I'm not saying it's easy. My point is that I have a fair amount of trust in "human decision theory" working out pretty well in most cases in the long run with enough wisdom. I even think something like this is worth trying with present-day AGI researchers (what I call "confrontation-worthy empathy"), though that is hard mode because you have so much less access.

1CodeForcer8mo

There's an important point to be made here that Hitler was not a genius, and in general the most evil people in history don't correlate at all to being the smartest people in history. In fact, the smartest people in history generally seemed more likely to contribute positively to the development of humanity. I would posit it's easier to make a high IQ child good for society, with positive nurturing. The alignment problem perhaps is thus less difficult with "super babies", because they can more easily see the irrationality in poor ethics and think better from first principles, being grounded in the natural alignment that comes from the fact we are all humans with similar sentience (as opposed to AI which might as well be a different species altogether). Given that Hitler's actions resulted in his death and the destruction of Germany, a much higher childhood IQ might even have blunted his evil. Also don't buy the idea that very smart humans automatically assume control. I suspect Kamala, Biden, Hillary, etc all had a higher IQ than Donald Trump, but he became the most powerful person on the planet.

0the gears to ascension8mo

My estimate is 97% not sociopaths, but only about 60% inclined to avoid teaming up with sociopaths. Germline engineering likely destroys most of what we're trying to save, via group conflict effects. There's a reason it's taboo.

4TsviBT8mo

Does the size of this effect, according to you, depend on parameters of the technology? E.g. if it clearly has a ceiling, such that it's just not feasible to make humans who are in a meaningful sense 10x more capable than the most capable non-germline-engineered human? E.g. if the technology is widespread, so that any person / group / state has access if they want it?

1kman8mo

My interpretation is that you're 99% of the way there in terms of work required if you start out with humans rather than creating a de novo mind, even if many/most humans currently or historically are not "aligned". Like, you don't need very many bits of information to end up with a nice "aligned" human. E.g. maybe you lightly select their genome for prosociality + niceness/altruism + wisdom, and treat them nicely while they're growing up, and that suffices for the majority of them.

2Noosphere898mo

I'd actually maybe agree with this, though with the caveat that there's a real possibility you will need a lot more selection/firepower as a human gets smarter, because you lack the ability to technically control humans in the way you can control AIs.

2TsviBT8mo

Also true, though maybe only for O(99%) of people.

2Noosphere898mo

I'd probably bump that down to O(90%) at max, and this could get worse (I'm downranking based on the number of psychopaths/sociopaths and narcissists that exist).

3Ebenezer Dukakis8mo

The jailbreakability and other alignment failures of current AI systems are also incredibly small peanuts compared to AGI omnicide. Yet they're still informative. Small-scale failures give us data about possible large-scale failures. Are you thinking of people such as Sam Altman, Demis Hassabis, Elon Musk, and Dario Amodei? If humans are 99% aligned, how is it that we ended up in a situation where major lab leaders look so unaligned? MIRI and friends had a fair amount of influence to shape this situation and align lab leaders, yet they appear to have failed by their own lights. Why? When it comes to AI alignment, everyone on this site understands that if a "boxed" AI acts nice, that's not a strong signal of actual friendliness. The true test of an AI's alignment is what it does when it has lots of power and little accountability. Maybe something similar is going on for humans. We're nice when we're powerless, because we have to be. But giving humans lots of power with little accountability doesn't tend to go well. Looking around you, you mostly see nice humans. That could be because humans are inherently nice. It could also be because most of the people around you haven't been given lots of power with little accountability. Dramatic genetic enhancement could give enhanced humans lots of power with little accountability, relative to the rest of us. [Note also, the humans you see while looking around are strongly selected for, which becomes quite relevant if the enhancement technology is widespread. How do you think you'd feel about humanity if you lived in Ukraine right now?] I want to see actual, detailed calculations of p(doom) from supersmart humans vs supersmart AI, conditional on each technology being developed. Before charging ahead on this, I want a superforecaster-type person to sit down, spend a few hours, generate some probability estimates, publish a post, and request that others red-team their work. I don't feel like that is a lot to ask.

3TsviBT8mo

But you don't go from a 160 IQ person with a lot of disagreeability and ambition, who ends up being a big commercial player or whatnot, to 195 IQ and suddenly get someone who just sits in their room for a decade and then speaks gibberish into a youtube livestream and everyone dies, or whatever. The large-scale failures aren't feasible for humans acting alone. For humans acting very much not alone, like big AGI research companies, yeah that's clearly a big problem. But I don't think the problem is about any of the people you listed having too much brainpower. (I feel we're somewhat talking past each other, but I appreciate the conversation and still want to get where you're coming from.)

1Ebenezer Dukakis8mo

How about a group of superbabies that find and befriend each other? Then they're no longer acting alone. I don't think problems caused by superbabies would look distinctively like "having too much brainpower". They would look more like the ordinary problems humans have with each other. Brainpower would be a force multiplier. Thanks. I mostly just want people to pay attention to this problem. I don't feel like I have unique insight. I'll probably stop commenting soon, since I think I'm hitting the point of diminishing returns.

[-]TsviBT8mo105

I mostly just want people to pay attention to this problem.

Ok. To be clear, I strongly agree with this. I think I've been responding to a claim (maybe explicit, or maybe implicit / imagined by me) from you like: "There's this risk, and therefore we should not do this.". Where I want to disagree with the implication, not the antecedent. (I hope to more gracefully agree with things like this. Also someone should make a LW post with a really catchy term for this implication / antecedent discourse thing, or link me the one that's already been written.)

But I do strongly disagree with the conclusion "...we should not do this", to the point where I say "We should basically do this as fast as possible, within the bounds of safety and sanity.". The benefits are large, the risks look not that bad and largely ameliorable, and in particular the need regarding existential risk is great and urgent.

That said, more analysis is definitely needed. Though in defense of the pro-germline engineering position, there's few resources, and everyone has a different objection.

4Noosphere898mo

I will go further, and say the human universals are nowhere near strong enough to assume that alignment of much more powerful people will automatically/likely happen, or that not aligning them produces benevolent results, and the reason for this is humans are already misaligned, in many cases very severely to each other, so allowing human augmentation without institutional reform makes things a lot worse by default. It is better to solve the AI alignment problem first, then have a legal structure created by AIs that can make human genetic editing safe, rather than try to solve the human alignment problem: https://www.lesswrong.com/posts/DfrSZaf3JC8vJdbZL/how-to-make-superbabies#jgDtAPXwSucQhPBwf

[-]LWLW8mo160

I honestly think the EV of superhumans is lower than the EV for AI. sadism and wills to power are baked into almost every human mind (with the exception of outliers of course). force multiplying those instincts is much worse than an AI which simply decides to repurpose the atoms in a human for something else. i think people oftentimes act like the risk ends at existential risks, which i strongly disagree with. i would argue that everyone dying is actually a pretty great ending compared to hyperexistential risks. it is effectively +inf relative utility.

with AIs we're essentially putting them through selective pressures to promote benevolence (as a hedge by the labs in case they don't figure out intent alignment). that seems like a massive advantage compared to the evolutionary baggage associated with humans.

with humans you'd need the will and capability to engineer in at least +5sd empathy and -10sd sadism into every superbaby. but people wouldn't want their children to make them feel like shitty people so they would want them to "be more normal."

1kman8mo

I don't think the result of intelligence enhancement would be "multiplying those instincts" for the vast majority of people; humans don't seem to end up more sadistic as they get smarter and have more options. I'm curious what value you assign to the ratio [U(paperclipped) - U(worst future)] / [U(best future) - U(paperclipped)]? It can't be literally infinity unless U(paperclipped) = U(best future). So your model is that we need to eradicate any last trace of sadism before superbabies is a good idea?

2Mateusz Bagiński8mo

Artificial wombs may remove this bottleneck.

6TsviBT8mo

No I mean like a person can't 10x their compute.

8LWLW8mo

I’m sure you’ve already thought about this, but it seems like the people who would be willing and able to jump through all of the hoops necessary would likely have a higher propensity towards power-seeking and dominance. So if you don’t edit the personality as well, what was it all for besides creating a smarter god-emperor? I think that in the sane world you’ve outlined where people deliberately avoid developing AGI, an additional level of sanity would be holding off on modifying intelligence until we have the capacity to perform the personality edits to make it safe. I can just imagine this turning into a world where the rich who are able to make their children superbabies compete with the rest of the elite over whose child will end up ruling the world. I’m sorry but I’d rather be turned into paper-clips then live in a world where a god-emperor can decide to torture me with their AGI-slave for the hell of it. How is that a better world for anyone but the god-emperor? But people are so blind and selfish, they just assume that they or their offspring would be god-emperor. At least with AI people are scared enough that they’re putting focused effort into trying to make it nice. People won’t put that much effort into their children. I mean hell, figuring out personality editing would probably just make things backfire. People would choose to make their kids more ruthless, not less.

9GeneSmith8mo

It's a fair concern. But the problem of predicting personality can be solved! We just need more data. I also worry somewhat about brilliant psychopaths. But making your child a psychopath is not necessarily going to give them an advantage. Also can you imagine how unpleasant raising a psychopath would be? I don't think many parents would willingly sign up for that.

8LWLW8mo

I certainly wouldn’t sign up to do that, but the type of individual I’m concerned about likely wouldn’t mind sacrificing nannies if their lineage could “win” in some abstract sense. I think it’s great that you’re proposing a plan beyond “pray the sand gods/Sam Altman are benevolent.” But alignment is going to be an issue for superhuman agents, regardless of if they’re human or not.

[-]GeneSmith8mo111

Agreed. I've actually had a post in draft for a couple of years that discusses some of the paralleles between alignment of AI agents and alignment of genetically engineered humans.

I think we have a huge advantage with humans simply because there isn't the same potential for runaway self-improvement. But in the long term (multiple generations), it would be a concern.

9Ebenezer Dukakis8mo

If you look at the grim history of how humans have treated each other on this planet, I don't think it's justified to have a prior that this is gonna go well. Humans didn't have the potential for runaway self-improvement relative to apes. That was little comfort for the apes.

7LWLW8mo

That sounds very interesting! I always look forward to reading your posts. I don’t know if you know any policy people, but in this world, it would need to be punishable by jail-time to genetically modify intelligence without selecting for pro-sociality. Any world where that is not the case seems much, much worse than just getting turned into paper-clips.

4Noosphere898mo

I think the runaway self-improvement problem is IMO vastly outweighed by other problems on aligning humans, like the fact that any control technique on AI would be illegal because of it being essentially equivalent to brainwashing, such that I consider AIs much more alignable than humans, and I think the human intelligence augmentation path is way more risky and fraught than people think for alignment purposes.

9LWLW8mo

I agree. At least I can laugh if the AGI just decides it wants me as paperclips. There will be nothing to laugh about with ruthless power-seeking humans with godlike power.

2kman8mo

A lot of (most?) humans end up nice without needing to be controlled / "aligned", and I don't particularly expect this to break if they grow up smarter. Trying to control / "align" them wouldn't work anyway, which is also what I predict will happen with sufficiently smart AI.

6Noosphere898mo

I think this is my disagreement, in that I don't think that most humans are in fact nice/aligned to each other by default, and the reason why this doesn't lead to catastrophe broadly speaking is a combo of being able to rely on institutions/mechanism design that means even if people are misaligned, you can still get people well off under certain assumptions (capitalism and the rule of law being one such example), combined with the inequalities not being so great that individual humans can found their own societies, except in special cases. Even here, I'd argue that human autocracies are very often misaligned to their citizens values very severely. To be clear about what I'm not claiming, I'm not saying that alignment is worthless, or alignment always or very often fails, because it's consistent with a world where >50-60% of alignment attempts are successful. This means I'm generally much more scared of very outlier smart humans, for example a +7-12 SD human that was in power of a large group of citizens, assuming no other crippling disabilities unless they are very pro-social/aligned to their citizenry. I'm not claiming that alignment will not work, or even that will very often not work, but rather that the chance of failure is real and the stakes are quite high long-term. (And that's not even addressing how you could get super-smart people to work on the alignment problem).

4LWLW8mo

This is just a definition for the sake of definition, but I think you could define a human as aligned if they could be given an ASI slave and not be an S-risk. I really think that under this definition, the absolute upper bound of “aligned” humans is 5%, and I think it’s probably a lot lower.

4Noosphere898mo

I'm more optimistic, in that the upper bound could be as high as 50-60%, but yeah the people in power are unfortunately not part of this, and I'd only trust 25-30% of the population in practice if they had an ASI slave.

1kman8mo

What would it mean for them to have an "ASI slave"? Like having an AI that implements their personal CEV?

1LWLW8mo

Yeah something like that, the ASI is an extension of their will.

2kman8mo

So you think that, for >95% of currently living humans, the implementation of their CEV would constitute an S-risk in the sense of being worse than extinction in expectation? This is not at all obvious to me; in what way do you expect their CEVs to prefer net suffering?

4kman8mo

I mean if we actually succeeded at making people who are +7 SD in a meaningful way, I'd expect that at least good chunk of them would figure out for themselves that it makes sense to work on it.

3Noosphere898mo

That requires either massive personality changes to make them more persuadable, or massive willingness of people to put genetic changes in their germline, and I don't expect either of these to happen before AI automates everything and either takes over, leaving us extinct or humans/other AI control/align AIs successfully. (A key reason for this is that Genesmith admitted that the breakthroughs in germline engineering can't transfer to the somatic side, and that means we'd have to wait 25-30 years in order for it to grow, minimum given that society won't maximally favor the genetically lucky, and that's way beyond most plausible AI timelines at this point)

1kman8mo

If they're that smart, why will they need to be persuaded?

2Noosphere898mo

Because they might consider that other problems are more worth their time, since smartness changes change their values little. And maybe they believe that AI alignment isn't impactful for technical/epistemic reasons. I'm confused/surprised I need to make this point, because I don't automatically think they will be persuaded that AI alignment is a big problem they will need to work on, and some effort will likely still need to be required.

1kman8mo

I mean if they care about solving problems at all, and we are in fact correct about AGI ruin, then they should predictably come to view it as the most important problem and start to work on it? Are you imagining they're super myopic or lazy and just want to think about math puzzles or something? If so, my reply is that even if some of them ended up like that, I'd be surprised if they all ended up like that, and if so that would be a failure of the enhancement. The aim isn't to create people who we will then carefully persuade to work on the problem, the aim is for some of them to be smart + caring + wise enough to see the situation we're in and decide for themselves to take it on.

2Noosphere898mo

More so that I'm imagining they might not even have heard of the argument, and it's helpful to note that people like Terence Tao, Timothy Gowers and more are all excellent people at their chosen fields, but most people that have a big impact on the world don't go into AI alignment. Remember, superintelligence is not omniscience. So I don't expect them to be self motivated to work on this specific problem without at least a little persuasion. I'd expect a few superintelligent adults to join alignment efforts, but nowhere near thousands or tens of thousands, and I'd upper bound it at 300-500 new researchers at most in 15-25 years. Much less impactful than automating AI safety.

3kman8mo

I don't think this will work.

0Noosphere898mo

How much probability do you assign to automating AI safety not working in time? Because I believe the preparing to automate AI safety work is probably the highest-value in pure ability to reduce X-risk probability, assuming it does work, so I assign much higher EV to automating AI safety, relative to other approaches.

1kman8mo

I think I'm at <10% that non-enhanced humans will be able to align ASI in time, and if I condition on them succeeding somehow I don't think it's because they got AIs to do it for them. Like maybe you can automate some lower level things that might be useful (e.g. specific interpretability experiments), but at the end of the day someone has to understand in detail how the outcome is being steered or they're NGMI. Not sure exactly what you mean by "automating AI safety", but I think stronger forms of the idea are incoherent (e.g. "we'll just get AI X to figure it all out for us" has the problem of requiring X to be aligned in the first place).

2Noosphere898mo

As far as what a plan to automate AI safety would work out in practice, assuming a relatively strong version of the concept is in this post below, and there will be another post that comes out by the same author talking more about the big risks discussed in the comments below: https://www.lesswrong.com/posts/TTFsKxQThrqgWeXYJ/how-might-we-safely-pass-the-buck-to-ai In general, I think the crux is that in most timelines (at a lower bound, 65-70%) that have AGI developed relatively soon (so timelines from 2030-2045, roughly), and the alignment problem isn't solvable by default/it's at least non-trivially tricky to solved, conditioning on alignment success looks more like "we've successfully figured out how to prepare for AI automation of everything, and we managed to use alignment and control techniques well enough that we can safely pass most of the effort to AI", rather than other end states like "humans are deeply enhanced" or "lawmakers actually coordinated to pause AI, and are actually giving funding to alignment organizations such that we can make AI safe."

2samuelshadrach8mo

How do you know you can afford to wait multiple generations? My guess is superhuman 6 year olds demonstrating their capabilities on YouTube is sufficient to start off an international arms race for more superhumans. (Increase number of people and increase capability level of each person.) And once the arms race is started it may never stop until the end state of this self-improvement is hit.

2kman8mo

Not at all obvious to me this is true. Do you mean to say a lot of people would, or just some small fraction, and you think a small fraction is enough to worry?

2LWLW8mo

I should have clarified, I meant a small fraction and that that is enough to worry.

[-]TsviBT8mo2518

After I finish my methods article, I want to lay out a basic picture of genomic emancipation. Genomic emancipation means making genomic liberty a right and a practical option. In my vision, genomic liberty is quite broad: it would include for example that parents should be permitted and enabled to choose:

to enhance their children (e.g. supra-normal health; IQ at the outer edges of the human envelope); and/or
to propagate their own state even if others would object (e.g. blind people can choose to have blind children); and/or
to make their children more normal even if there's no clear justification through beneficence (I would go so far as to say that, for example, parents can choose to make their kid have a lower IQ than a random embryo from the parents would be in expectation, if that brings the kid closer to what's normal).

These principles are more narrow than general genomic liberty ("parents can do whatever they please"), and I think have stronger justifications. I want to make these narrower "tentpole" principles inside of the genomic liberty tent, because the wider principle isn't really tenable, in part for the reasons you bring up. There are genomic choices that should ... (read more)

2kman8mo

In that case I'd repeat GeneSmith's point from another comment: "I think we have a huge advantage with humans simply because there isn't the same potential for runaway self-improvement." If we have a whole bunch of super smart humans of roughly the same level who are aware of the problem, I don't expect the ruthless ones to get a big advantage. I mean I guess there is some sort of general concern here about how defense-offense imbalance changes as the population gets smarter. Like if there's some easy way to destroy the world that becomes accessible with IQ > X, and we make a bunch of people with IQ > X, and a small fraction of them want to destroy the world for some reason, are the rest able to prevent it? This is sort of already the situation we're in with AI: we look to be above the threshold of "ability to summon ASI", but not above the threshold of "ability to steer the outcome". In the case of AI, I expect making people smarter differentially speeds up alignment over capabilities: alignment is hard and we don't know how to do it, while hill-climbing on capabilities is relatively easy and we already know how to do it. I should also note that we have the option of concentrating early adoption among nice, sane, x-risk aware people (though I also find this kind of cringe in a way and predict this would be an unpopular move). I expect this to happen by default to some extent.

6David Gross8mo

There are some promising but under-utilized interventions for improving personality traits / virtues in already-developed humans,* and a dearth of research about possible interventions for others. If we want more of that sort of thing, we might be better advised to fill in some of those gaps rather than waiting for a new technology and a new generation of megalopsychebabies.

2DanielLC8mo

Imagine Star Trek if Khan were also engineered to be a superhumanly moral person.

1Roger Scott8mo

Putting aside for the moment the fact that even "intelligence" is hardly a well-defined and easily quantified property, isn't it rather a giant leap to say we even know what a "better" personality is? I might agree that some disorders are reasonably well defined, and those might be candidates for trying to "fix", but if you're trying to match greater intelligence with "better" personality I think you first need a far better notion of what "better" personality actually means.

[-]momom28mo22-4

I'm surprised to see no one in the comments whose reaction is "KILL IT WITH FIRE", so I'll be that guy and make a case why this research should be stopped rather than pursued:

On the one hand, there is obviously enormous untapped potential in this technology. I don't have issues about the natural order of life or some WW2 eugenics trauma. From my (unfamiliar with the subject) eyes, you propose a credible way to make everyone healthier, smarter, happier, at low cost and within a generation, which is hard to argue against.

On the other hand, you spend no time mentioning the context in which this technology will be developed. I imagine there will be significant public backlash and that most advances on superbabies-making will be made by private labs funded by rich tech optimists, so it seems overwhelmingly likely to me that if this technology does get developed in the next 20 years, it will not improve everyone.

At this point, we're talking about the far future, so I need to make a caveat for AI: I have no idea how the new AI world will interact with this, but there are a few most likely futures I can condition on.

Everyone dies: No point talking about superbabies.
Cohabitive singlet

... (read more)

6Kaj_Sotala8mo

There's also the option that even if this technology is initially funded by the wealthy, learning curves will then drive down its cost as they do for every technology, until it becomes affordable for governments to subsidize its availability for everyone.

[-]kave8mo200

Previous discussion

[-]LGS8mo190

Are your IQ gain estimates based on plain GWAS or on family-fixed-effects-GWAS? You don't clarify. The latter would give much lower estimates than the former

7kman8mo

Plain GWAS, since there aren't any large sibling GWASes. What's the basis for the estimates being much lower and how would we properly adjust for them?

[-]LGS8mo11-3

Your OP is completely misleading if you're using plain GWAS!

GWAS is an association -- that's what the A stands for. Association is not causation. Anything that correlates with IQ (eg melanin) can show up in a GWAS for IQ. You're gonna end up editing embryos to have lower melanin and claiming their IQ is 150

9kman8mo

The IQ GWAS we used was based on only individuals of European ancestry, and ancestry principal components were included as covariates as is typical for GWAS. Non-causal associations from subtler stratification is still a potential concern, but I don't believe it's a terribly large concern. The largest educational attainment GWAS did a comparison of population and direct effects for a "cognitive performance" PGI and found that predicted direct (between sibling) effects were only attenuated by a factor of 0.824 compared to predicted population level effects. If anything I'd expect their PGI to be worse in this regard, since it included variants with less stringent statistical power cutoffs (so I'd guess it's more likely that non-causal associations would sneak in, compared to the GWAS we used).

7LGS8mo

You should decide whether you're using a GWAS on cognitive performance or on educational attainment (EA). This paper you linked is using a GWAS for EA, and finding that very little of the predictive power was direct effects. Exactly the opposite of your claim: Then they compare this to cognitive performance. For cognitive performance, the ratio was better, but it's not 0.824, it's 0.8242=0.68. But actually, even this is possibly too high: the table in figure 4 has a ratio that looks much smaller than this, and refers to supplementary table 10 for numbers. I checked supplementary table 10, and it says that the "direct-population ratio" is 0.656, not 0.824. So quite possibly the right value is 0.6562=0.43 even for cognitive performance. Why is the cognitive performance number bigger? Well, it's possibly because there's less data on cognitive performance, so the estimates are based on more obvious or easy-to-find effects. The final, predictive power of the direct effects for EA and for cognitive performance is similar, around 3% of the variance, if I'm reading it correctly (not sure about this). So the ratios are somewhat different, but the population GWAS predictive power is also somewhat different in the opposite direction, and these mostly cancel out.

9kman8mo

That's variance explained. I was talking about effect size attenuation, which is what we care about for editing. Supplementary table 10 is looking at direct and indirect effects of the EA PGI on other phenotypes. The results for the Cog Perf PGI are in supplementary table 13.

5LGS8mo

Thanks! I understand their numbers a bit better, then. Still, direct effects of cognitive performance explain 5% of variance. Can't multiply the variance explained of EA by the attenuation of cognitive performance! Do you have evidence for direct effects of either one of them being higher than 5% of variance? I don't quite understand your numbers in the OP but it feels like you're inflating them substantially. Is the full calculation somewhere?

4kman8mo

Not quite sure which numbers you're referring to, but if it's the assumed SNP heritability, see the below quote of mine from another comment talking about missing heritability for IQ: The h^2 = 0.19 estimate from this GWAS should be fairly robust to stratification, because of how the LDSC estimator works. (To back this up: a recent study that actually ran a small GWAS on siblings, based on the same cognitive test, also found h^2 = 0.19 for direct effects.)

4LGS8mo

The paper you called largest ever GWAS gave a direct h^2 estimate of 0.05 for cognitive performance. How are these papers getting 0.2? I don't understand what they're doing. Some type of meta analysis? The test-retest reliability you linked has different reliabilities for different subtests. The correct adjustment depends on which subtests are being used. If cognitive performance is some kind of sumscore of the subtests, its reliability would be higher than for the individual subtests. Also, I don't think the calculation 0.2*(0.9/0.6)^2 is the correct adjustment. A test-retest correlation is already essentially the square of a correlation of the test with an underlying latent factor (both the test AND the retest have error). E.g. if a test T can be written as T=aX+sqrt(1-a)E where X is ability and E is error (all with standard deviation 1 and the error independent of the ability), then a correlation of T with a resample of T (with new independent error but same ability) would be a^2. But the adjustment to h^2 should be proportional to a^2, so it should be proportional to the test-retest correlation, not the square of the test-retest correlation. Am I getting this wrong?

5kman8mo

You're mixing up h^2 estimates with predictor R^2 performance. It's possible to get an estimate of h^2 with much less statistical power than it takes to build a predictor that good. "Fluid IQ" was the only subtest used. Good catch, we'll fix this when we revise the post.

6LGS8mo

Thanks. I understand now. But isn't the R^2 the relevant measure? You don't know which genes to edit to get the h^2 number (nor do you know what to select on). You're doing the calculation 0.2*(0.9/0.6)^2 when the relevant calculation is something like 0.05*(0.9/0.6). Off by a factor of 6 for the power of selection, or sqrt(6)=2.45 for the power of editing

3kman8mo

Not for this purpose! The simulation pipeline is as follows: the assumed h^2 and number of causal variants is used to generate the genetic effects -> generate simulated GWASes for a range of sample sizes -> infer causal effects from the observed GWASes -> select top expected effect variants for up to N (expected) edits.

6LGS8mo

I'm talking about this graph: What are the calculations used for this graph. Text says to see the appendix but the appendix does not actually explain how you got this graph.

2kman8mo

This is based on inferring causal effects conditional on this GWAS. The assumed heritability affects the prior over SNP effect sizes.

5LGS8mo

I don't understand. Can you explain how you're inferring the SNP effect sizes?

3kman8mo

With a method similar to this. You can easily compute the exact likelihood function P(GWAS results | SNP effects), which when combined with a prior over SNP effects (informed by what we know about the genetic architecture of the trait) gives you a posterior probability of each SNP being causal (having nonzero effect), and its expected effect size conditional on being causal (you can't actually calculate the full posterior since there are 2^|SNPs| possible combinations of SNPs with nonzero effects, so you need to do some sort of MCMC or stochastic search). We may make a post going into more detail on our methods at some point.

[-]LGS8mo134

You should show your calculation or your code, including all the data and parameter choices. Otherwise I can't evaluate this.

I assume you're picking parameters to exaggerate the effects, because just from the exaggerations you've already conceded (0.9/0.6 shouldn't be squared and attenuation to get direct effects should be 0.824), you've already exaggerated the results by a factor of sqrt(0.9/0.6)/0.824 for editing, which is around a 50% overestimate.

I don't think that was deliberate on your part, but I think wishful thinking and the desire to paint a compelling story (and get funding) is causing you to be biased in what you adjust for and in which mistakes you catch. It's natural in your position to scrutinize low estimates but not high ones. So to trust your numbers I'd need to understand how you got them.

[-]GeneSmith8mo112

There is one saving grace for us which is that the predictor we used is significantly less powerful than ones we know to exist.

I think when you account for both the squaring issue, the indirect effect things, and the more powerful predictors, they're going to roughly cancel out.

Granted, the more powerful predictor itself isn't published, so we can't rigorously evaluate it either which isn't ideal. I think the way to deal with this is to show a few lines: one for the "current publicly available GWAS", one showing a rough estimate of the gain using the privately developed predictor (which with enough work we could probably replicate), and then one or two more for different amounts of data.

All of this together WILL still reduce the "best case scenario" from editing relative to what we originally published (because with the better predictor we're closer to "perfect knowledge" than where we were with the previous predictor.

At some point we're going to re-run the calculations and publish an actual proper writeup on our methodology (likely with our code).

Also I just want to say thank you for taking the time to dive deep into this with us. One of the main reasons I post on LessWrong is because there is such high quality feedback relative to other sites.

5kman8mo

The code is pretty complicated and not something I'd expect a non-expert (even a very smart one) to be able to quickly check over; it's not just a 100 line python script. (Or even a very smart expert for that matter, more like anyone who wasn't already familiar with our particular codebase.) We'll likely open source it at some point in the future, possibly soon, but that's not decided yet. Our finemapping (inferring causal effects) procedure produces ~identical results to the software from the paper I linked above when run on the same test data (though we handle some additional things like variable per-SNP sample sizes and missing SNPs which that finemapper doesn't handle, which is why we didn't just use it). The parameter choices which determine the prior over SNP effects are the number of causal SNPs (which we set to 20,000) and the SNP heritability of the phenotype (which we set to 0.19, as per the GWAS we used). The erroneous effect size adjustment was done at the end to convert from the effect sizes of the GWAS phenotype (low reliability IQ test) to the effect sizes corresponding to the phenotype we care about (high reliability IQ test). We want to publish a more detailed write up of our methods soon(ish), but it's going to be a fair bit of work so don't expect it overnight. Yep, fair enough. I've noticed myself doing this sometimes and I want to cut it out. That said, I don't think small-ish predictable overestimates to the effect sizes are going to change the qualitative picture, since with good enough data and a few hundred to a thousand edits we can boost predicted IQ by >6 SD even with much more pessimistic assumptions, which probably isn't even safe to do (I'm not sure I expect additivity to hold that far). I'm much more worried about basic problems with our modelling assumptions, e.g. the assumption of sparse causal SNPs with additive effects and no interactions (e.g. what if rare haplotypes are deleterious due to interactions that don't show up in GW

[-]AnthonyC8mo189

I definitely want to see more work in this direction, and agree that improving humans is a high-value goal.

But to play devil's advocate for a second on what I see as my big ethical concern: There's a step in the non-human selective breeding or genetic modification comparison where the experimenter watches several generations grow to maturity, evaluates whether their interventions worked in practice, and decides which experimental subjects if any get to survive or reproduce further. What's the plan for this step in humans, since "make the right prediction every time at the embryo stage" isn't a real option? '

Concrete version of that question: Suppose we implement this as a scalable commercial product and find out that e.g. it causes a horrible new disease, or induces sociopathic or psychopathic criminal tendencies, that manifest at age 30, after millions of parents have used it. What happens next?

[-]GeneSmith8mo125

I think almost everyone misunderstands the level of knowledge we have about what genetic variants will do.

Nature has literally run a randomized control trial for genes already. Every time two siblings are created, the set of genes they inherit from each parent are scrambled and (more or less) randomly assigned to each. That's INCREDIBLY powerful for assessing the effects of genes on life outcomes. Nature has run a literal multi-generational randomized control trial for the effect of genes on everything. We just need to collect the data.

This gives you a huge advantage over "shot-in-the-dark" type interventions where you're testing something without any knowledge about how it performs over the long run.

Also, nature is ALREADY running a giant parallelized experiment on us every time a new child is born. Again, the genes they get from their parents are randomized. If reshuffling genetic variants around were extremely dangerous we'd see a huge death rate in the newborn population. But that is not in fact what we see. You can in fact change around some common genetic variants without very much risk.

And if you have a better idea about what those genes do (which we increasingly do), then you can do even better.

There are still going to be risks, but the biggest ones I actually worry about are about getting the epigenetics right.

But there we can just copy what nature has done. We don't need to modify anything.

4AnthonyC8mo

True, and this does indicate that children produced from genes found in 2 parents will not be outside the range which a hypothetical natural child of theirs could occupy. I am also hopeful that this is what matters, here. However, there are absolutely, definitely viable combinations of genes found in a random pair of parents which, if combined in a single individual, result in high-IQ offspring predisposed to any number of physical or mental problems, some of which may not manifest until long after the child is born. In practice, any intervention of the type proposed here seems likely to create many children with specific combinations of genes which we know are individually helpful for specific metrics, but which may not often (or ever) have all co-occurred. This is true even in the cautious, conservative early generations where we stay within the scope of natural human variations. Thereafter, how do we ensure we're not trialing someone on an entire generation at once? I don't want us to end up in a situation where a single mistake ends up causing population-wide problems because we applied it to hundreds of millions of people before the problem manifested.

5TsviBT8mo

This is a good argument for not going outside the human envelope in one shot. But if you're firmly within the realm where natural human genomes are, we have 8 billion natural experiments running around, some of which are sibling RCTs.

3lemonhope8mo

GeneSmith forgot to explicitly say that you can and should weight against sociopathy. Parents will be motivated to do this because if your kid is a jerk then your life will be miserable. (I do think if you select for success without selecting against sociopathy then you'll get lots of sociopaths.) I would bet against some weird disease manifesting, especially if you are weighting for general health.

4AnthonyC8mo

And that makes perfect sense. I guess I'm just not sure I trust any particular service provider or research team to properly list the full set of things it's important to weight against. Kind of feels like a lighter version of not trusting a list of explicit rules someone claims will make an AI safe.

[-]Metacelsus8mo140

Ovelle, who is planning to use growth and transcription factors to replicate key parts of the environment in which eggs are produced rather than grow actual feeder cells to excrete those factors. If it works, this approach has the advantage of speed; it takes a long time to grow the feeder cells, so if you can bypass some of that you can make eggs more quickly. Based on some conversations I’ve had with one of the founders I think $50 million could probably accelerate progress by about a year.

A few comments on this:
1. The "feeder cells" you're discussing here are from the method in this paper from the Saitou lab, who used feeder cells to promote development of human PGC-like cells to oogonia. But "takes a long time to grow the feeder cells" is not the issue. In fact, the feeder cells are quite easy to grow. The issue is that it takes a long time for PGC-like cells to develop to eggs, if you're strictly following the natural developmental trajectory.

2. The $50 million number is for us to set up our own nonhuman primate research facility, which would accelerate our current trajectory by *approximately* a year. On our current trajectory, we are going to need to raise about $5-10 million in the near future to scale up our research. We have already raised $2.15 million and we will start fundraising again this summer. But it's not like we need $50 million to make progress (although it would certainly help!)

6GeneSmith8mo

Thanks for the clarification. I'll amend the original post.

5lemonhope8mo

Could you do all the research on a boat in the ocean? Excuse the naive question.

[-]Metacelsus8mo131

On the topic of SuperSOX and how it relates to making eggs from stem cells:

The requirement for an existing embryo (to transfer the edited stem cells into) means that having an abundant source of eggs is important for this method, both for optimizing the method by screening many conditions, and for eventual use in the clinic.

So, in vitro oogenesis could play a key role here.

For both technologies, I think the main bottleneck right now is nonhuman primate facilities for testing.

Finally: we need to be sure not to cause another He Jiankui event (where an irresponsible study resulted in a crackdown on the field). Epigenetic issues could cause birth defects, and if this happens, it will set back the field by quite a lot. So safety is important! Nobody cares if their baby has the genes for 200 IQ, if the baby also has Prader-Willi syndrome.

[-]B. P.8mo120

Thank you, you inspired me to work on embryo selection in my home country!

4GeneSmith8mo

Cool! Are you working for an existing company or are you starting your own?

6B. P.8mo

I considered making my own startup, but then i learned that one lab is trying to get into this field so i'm aiming to join them. After squeezing out everything from embryo selection i certainly will try gene editing, so let's see who gets superbabies first :) From Russia with love (hope that'll motivate someone)

[-]Rosoe8mo112

May I ask what your respective scientific and genetics backgrounds are? I ask because this piece reads like many pieces where enthusiastic non-scientists pull together a bunch of papers without being able to assess the quality of those papers. I've also noticed a behaviour where you can't assess the quality of a paper then you over-estimate the positive effects that confirm your beliefs and ignore potential negative effects. This is an odd approach for most biologists I've worked with but is pretty typical of a layman. You also happy to work with single papers for many of your ideas, which again is something laymen inexperienced with scientific problems would do.

1GeneSmith8mo

I'd be interested in hearing where specifically you think we are doing that.

1Rosoe8mo

You’d like me to point out where you’ve only referenced individual papers for particular ideas and concepts? I can do that but I’d appreciate an answer to my question first if that’s okay.

1RichardJActon8mo

GeneSmith gave some more details about his background in this episode of the Bayesian Conspiracy podcast: https://www.thebayesianconspiracy.com/2025/02/231-superbabies-with-gene-smith/

4Rosoe8mo

If you have listened to the episode it would be nice to relay what those details were. I'm not so interested in listening to a 2 hour podcast mostly about what I just read above to get a few sentences of detail especially so when Gene is here! I'm also keen to know about Kman's background too.

[-]momom28mo114

I'm having trouble understanding your ToC in a future influenced by AI. What's the point of investigating this if it takes 20 years to become significant?

7GeneSmith8mo

Kman and I probably differ somewhat here. I think it's >90% likely that if we continue along the current trajectory we'll get AGI before the superbabies grow up. This technology only starts to become really important if there's some kind of big AI disaster or a war that takes down most of the world's chip fabs. I think that's more likely than people are giving it credit for and if it happens this will become the most important technology in the world. Gene editing research is much less centralized than chip manufacturing. Basically all of the research can be done in normal labs of the type seen all over the world. And the supply chain for reagents and other inputs is much less centralized than the supply chain for chip fabrication. You don't have a hundred billion dollar datacenter than can be bombed by hypersonic projectiles. The research can happen almost anywhere. So this stuff is just naturally a lot more robust than AI in the event of a big conflict.

6kman8mo

I mostly think we need smarter people to have a shot at aligning ASI, and I'm not overwhelmingly confident ASI is coming within 20 years, so I think it makes sense for someone to have the ball on this.

1momom28mo

In that case, per my other comment, I think it's much more likely that superbabies concern only a small fraction of the population and exacerbates inequality without bringing the massive benefits that a generally more capable population would. Do you think superbabies would be put to work on alignment in a way that makes a difference due to geniuses driving the field? I'm having trouble understanding how concretely you think superbabies can lead to significantly improved chance of helping alignment.

4kman8mo

My guess is that peak intelligence is a lot more important than sheer numbers of geniuses for solving alignment. At the end of the day someone actually has to understand how to steer the outcome of ASI, which seems really hard and no one knows how to verify solutions. I think that really hard (and hard to verify) problem solving scales poorly with having more people thinking about it. Sheer numbers of geniuses would be one effect of raising the average, but I'm guessing the "massive benefits" you're referring to are things like coordination ability and quality of governance? I think those mainly help with alignment via buying time, but if we're already conditioning on enhanced people having time to grow up I'm less worried about time, and also think that sufficiently widespread adoption to reap those benefits would take substantially longer (decades?).

4GeneSmith8mo

It's possible I'm misunderstanding your comment, so please correct me if I am, but there's no reason you couldn't do superbabies at scale even if you care about alignment. In fact, the more capable people we have the better. Kman may have his own views, but my take is pretty simple; there are a lot of very technically challenging problems in the field of alignment and it seems likely smarter humans would have a much higher chance of solving them.

[-]Sodium8mo1112

One thing not mentioned here (and I think should be talked about more) is that the naturally occurring genetic distribution is very unequal in a moral sense. A more egalitarian society would put a stop to Eugenics Performed by a Blind, Idiot God.

Have your doctor ever asked about if you have a family history of [illness]? For so many diseases, if your parents have it, you're more likely to have it, and your kids are more likely to have it. These illnesses plague families for generations.

I have a higher than average chance of getting hypertension. Without technology, so will my future kids. With gene editing, we can just stop that, once and for all. A just world is a world where no child is born predetermined to endure avoidable illness simply because of ancestral bad luck.

6GeneSmith8mo

Agreed, though unfortunately it's going to take a while to make this tech available to everyone. Also, if you want to prevent your children from getting hypertension, you can already do embryo selection right now! The reduction isn't always as large as what you can get for gene editing, but it's still noticeable. And it stacks generation after generation; your kids can use embryo selection to lower THEIR children's disease risk even more.

3David James8mo

In clear-cut cases, this principle seems sound; if a certain gene only has deleterious effects, and it can be removed, this is clearly better (for the individual and almost certainly for everyone else too). In practice, this becomes more complicated if one gene has multiple effects. (This may occur on its own or because the gene interacts with other genes.) What if the gene in question is a mixed bag? For example, consider a gene giving a 1% increased risk of diabetes while always improving visual acuity. To be clear, I'm saying complicated not unresolvable. Such tradeoffs can indeed be resolved with a suitable moral philosophy combined with sufficient data. However, the difference is especially salient because the person deciding isn't the person that has to live with said genes. The two people may have different philosophies, risk preferences, or lifestyles.

[-]Adele Lopez8mo108

However, the difference is especially salient because the person deciding isn't the person that has to live with said genes. The two people may have different moral philosophies and/or different risk preferences.

A good rule might be that the parents can only select alleles that one or the other of them have, and also have the right to do so as they choose, under the principle that they have lived with it. (Maybe with an exception for the unambiguously bad alleles, though even in that case it's unlikely that all four of the parent's alleles are the deleterious one or that the parents would want to select it.) Having the right to select helps protect from society/govt imposing certain traits as more or less valuable, and keeping within the parent's alleles maintains inheritance, which I think are two of the most important things people opposed to this sort of thing want to protect.

4TsviBT8mo

My current guess at the best regulatory stance--the one that ought to be acceptable to basically everyone, and that would result in good outcomes--is significantly more permissive, i.e. giving more genomic liberty to the parents. There should be rights to not genomically engineer at all, or to only GE along certain dimensions; and rights to normalize, or to propagate ones genes or traits, or to benefit the child, or to benefit others altruistically. Your rule might for example practically prevent a deaf couple from intentionally having a child who is deaf but otherwise normal. E.g. imagine if the couple's deafness alleles also carry separate health risks, but there are other deafness alleles that the couple does not have but that lead to deafness without other health risks. I still haven't fully thought through the consent objection, though. Restrictions on genomic liberty should be considered very costly: they break down walls against eugenics-type forces (i.e. forces on people's reproduction coming from state/collective power, and/or aimed at population targets). Like with other important values, this isn't 100% absolute. E.g. parents shouldn't be allowed to GE their children in order to make their children suffer a lot, or in a way that has a very high risk of creating a violent psychopath. But every such restriction rightfully invokes a big pile of "Wait, who decides what counts as a 'good' allele or as a 'disease'?". https://www.lesswrong.com/posts/DfrSZaf3JC8vJdbZL/how-to-make-superbabies?commentId=ZeranH3yDBGWNxZ7h

2Adele Lopez8mo

I think mostly we're on the same page then? Parents should have strong rights here, and the state should not. I think that there's enough variance within individuals that my rule does not practically restrict genomic liberty much, while making it much more palatable to the average person. But maybe that's wrong, or it still isn't worth the cost. That's a good point, I wouldn't want to prevent that. I'm not sure how likely this is to practically come up though. Strong agree.

6TsviBT8mo

This should be true for any trait that is highly polygenic and that we know many associated variants for, yeah. IDK, but if I had to make a guess I would guess that it's quite rare but does occur. Another sort of example might be: say there's a couple whose child will likely get some disease D. Maybe the couple has a very high genetic predisposition for D that can't be attenuated enough using GE, or maybe it's a non-genetic disease that's transmissible. And say there's a rare variant that protects against D (which neither parent has). It would be a risk, and potentially a consent issue, to experiment with editing in the rare variant; but it might be good all things considered. (If this sounds like sci-fi, consider that this is IIUC exactly the scenario that happened with the first CRISPR-edited baby! In that case there were multiple methodological issues, and the edit itself might have been a bad idea even prospectively, but the background scenario was like that.)

-4[comment deleted]8mo

[-]LGS8mo101

And these changes in chickens are mostly NOT the result of new mutations, but rather the result of getting all the big chicken genes into a single chicken.

Is there a citation for this? Or is that just a guess

[-]PipFoweraker8mo102

Thanks for the write-up, I recall a conversation introducing me to all these ideas in Berkeley last year and it's going to be very handy having a resource to point people at (and so I don't misremember details about things like the Yamanaka factors!).

Am I reading the current plan correctly such that the path is something like:
Get funding -> Continue R+D through primate trials -> Create an entity in a science-friendly, non-US state for human trials -> first rounds of Superbabies? That scenario seems like it would require a bunch of medical tourism, which I imagine is probably not off the table for people with the resources and mindset willing to participate in this.

[-]GeneSmith8mo202

Yes, that's more or less the plan. I think it's pretty much inevitable that the United States will fully legalize germline gene editing at some point. It's going to be pretty embarassing if rich American parents are flying abroad to have healthier children.

You can already see the tide starting to turn on this. Last Month Nature actually published an article about germline gene editing. That would NEVER have happened even just a few years ago.

When you go to CRISPR conferences on gene editing, many of the scientists will tell you in private that germline gene editing makes sense. But if you ask them to go on the record as supporting it publicly, they will of course refuse.

At some point there's going to be a preference cascade. People are going to start wondering why the US government is blocking technology to its future citizens healthier, happier, and smarter.

[-]Anrol L7mo96

First, sorry in advance for the annoying unimportant nitpick and thank you for this exquisite resource.

too busy with their washing machines to think about light bulbs or computers

I think washing machines are not the best metaphor. They have actually contributed quite considerably to global economic growth in the past half century, as researched and explained by Economics Explained.

Perhaps something more like electric heated seats would be more fitting?

2kman7mo

I agree with this critique; I think washing machines belong on the "light bulbs and computers" side of the analogy. The analogy has the form: "germline engineering for common diseases and important traits" : "gene therapy for a rare disease" :: "widespread, transformation uses of electricity" : x So x should be some very expensive, niche use of electricity that provides a very large benefit to its tiny user base (and doesn't arguably indirectly lead to large future benefits, e.g. via scientific discovery for a niche scientific instrument).

2GeneSmith7mo

I mean... I think adult gene therapy is great! It can cure diseases and provide treatments that are otherwise impossible. So I think it's more impactful than heated seats.

[-]TsviBT8mo92

We’ve successfully extracted and cultured human spermatogonial stem cells back in 2020.

To be clear, this paper AFAICT did not directly measure epigenomic state, and specifically imprinting state. (Which makes sense given what they were trying to do. In theory the imprints might show up in their transcriptomics, but who knows, and they didn't report that.) We don't know whether imprinting would be maintained for a great length of time in culture with their method.

[-]kave8mo90

Curated. Genetically enhanced humans are my best guess for how we achieve existential safety. (Depending on timelines, they may require a coordinated slowdown to work). This post is a pretty readable introduction to a bunch of the why and how and what still needs to be down.

I think this post is maybe slightly too focused on "how to genetically edit for superbabies" to fully deserve its title. I hope we get a treatment of more selection-based methods sometime soon.

GeneSmith mentioned the high-quality discussion as a reason to post here, and I'm glad we're a... (read more)

5GeneSmith8mo

Yes, the two other approaches not really talked about in this thread that could also lead to superbabies are iterated meiotic selection and genome synthesis. Both have advantages over editing (you don't need to have such precise knowledge of causal alleles with iterated meiotic selection or with genome synthesis), but my impression is they're both further off than an editing approach. I'd like to write more about both in the future.

[-]dmac_938mo90

Your introduction describes how I feel about my area of expertise too!

Working in the field of *neuroscience* is a bizarre experience. No one seems to be interested in the most interesting applications of their research. Neuroscience has significantly advanced in the past few decades. I keep telling people that soon we're all going to have self driving cars powerd by a horse's brain in a glass jar. Most people just laugh in disbelief, others make frightened noises and mention "ethical issues" or change the subject. The smart money is off chasing the deep learning pipe dream, and as a direct consequence, there is low-hanging fruit absolutely everywhere.

8JenniferRM8mo

Wait what? This feels "important if true" but I don't think it is true. I can think of several major technical barriers to the feasibility of this. To pick one... How do you feed video data into a brain? The traditional method would have involved stimulating neurons with the pixels captured electronically, but the clumsy stimulation process to transduce the signal into the brain itself would harm the stimulated neurons and not be very dense, so the brain would have low res vision, until the stimulated neurons die in less than a few months. Or at least, that was the model I've had for the last... uh... 10 years? Were major advances made when I wasn't looking?

5dmac_938mo

Sorry to get your hopes up but I was being facetious and provocative. Instead of a glass jar, our horse's brain is going to live inside of a computer simulation. Nonetheless, I think my argument still holds true. Neuroscientists scoff at the thought of whole brain simulation. They're incredulous and as a result they're unambitious. They want it but they know they can't have it; they've got sour grapes. Despite these bad vibes, they have been working diligently and I think we're not too far off from making simulations which are genuinely useful. ---------------------------------------- On a wacky side note, IMO, if we did have a horses brain in a jar, then interacting with it would be the easy part. There have been some really neat advances in how we interact with brains. * We can make neurons light up when they activate, see GCaMP * And here is a video of GCaMP in action: * We can activate synapses with light, see Optogenetics The hard part would be keeping it alive for its 25-30 year lifespan even though it's missing important internal organs like the heart, lungs, liver, and adaptive immune system.

[-]Purplehermann8mo90

Have you thought about how to get the data yourself?

Perhaps offering payment to people willing to get iq tested and give a genetic sample, and paying more for higher scores on the test?

I understand that money is an issue, but as long as you're raising this seems like an area you could plug infinite money into and get returns

[-]Walkabout8mo7-20

I feel that this project would be unethical to undertake, and I will try to explain why I get that reading.

It's not that I think using genetic engineering on children is categorically wrong. Mutations occur in every new human, and adding in some that seem likely to come in handy later is something one can make an argument for. A person's genome is the toolbox their body has to deal with the world, and it might be right to stock it with more tools.

But I think it is wrong to instrumentalize children in this way. If you go through many rounds of editing, each... (read more)

[-]GeneSmith8mo157

I think many people in academia more or less share your viewpoint.

Obviously genetic engineering does add SOME additional risk of people coming to see human children like commodities, but in my view it's massively outweighed by the potential benefits.

you end up with a child whose purpose is to fulfill the parameters of their human designers

I think whether or not people (and especially parents) view their children this way depends much more on cultural values and much less on technology.

There are already some parents who have very specific goals in mind for their children and work obsessively to realize them. This doesn't always work that well, and I'm not sure it will work that well even with genetic engineering.

Sure we will EVENTUALLY be able to shape personality better with gene editing (though I would note we don't really have the ability to do so currently), but human beings are very complicated. Gene editing is a fairly crude tool for shaping human behavior. You can twist the knobs for dozens of human traits, but I think anyone trying to predetermine their child's future is going to be disappointed.

The tremendous effort involved in trying to fit the child to the design pa

... (read more)

[-]Said Achmiz8mo1010

A person with HIV and a person without are both worth unity. These are fundamental results in disability studies.

Where are these studies that have results which are object-level ethical claims…? This seems not just improbable, but outright incoherent. Do you have any links to studies like this?

9TsviBT8mo

I agree with this as a significant thing to keep in mind, and have written about it here: https://berkeleygenomics.org/articles/Potential_perils_of_germline_genomic_engineering.html#objectification I think a pretty core lesson from this concern is that communication to parents is very important. Parents should understand: 1. What the traits do and don't mean that they are selecting for, including plausible consequences. 2. What uncertainties exist in the PGSes that are being used (generally lots of uncertainty), e.g. are they accidentally tracking something else as well, or might they perform less well than expected. 3. How much variation is still being left up to chance or environment; pointing out important things that aren't being tracked. 4. That overall the methods will have uncertain outcomes. 5. How to raise kids well regardless of their genomic foundation (i.e. cultural tech for parenting so your kids flourish). 6. That, at least in the scheme of genetic variation, the nudges applied by germline genomic engineering are a drop in the bucket. I agree with this and commented this on a draft. It's not a good way of thinking of germline engineered kids, and inaccurately implies there's some gradation and some single direction of desirability or superiority. I agree with this and agree it's pretty crucial, and possibly threatened by germline engineering, and possibly threatened by thinking of them as "super". I've written about this a bit here: https://berkeleygenomics.org/articles/Potential_perils_of_germline_genomic_engineering.html#loss-of-human-dignity Right, also true. And we (society) should be oriented around not abandoning people who become less typical because of germline engineering. https://berkeleygenomics.org/articles/Potential_perils_of_germline_genomic_engineering.html#centrifugal-force-on-marginalized-people https://berkeleygenomics.org/articles/Potential_perils_of_germline_genomic_engineering.html#erasure-of-some-kinds-of-people Also,

3rey gomez7mo

Not providing a child with the ability to think as deeply and as far as possible is also unethical.

[-]lemonhope8mo7-2

Someone please tell Altman and Musk they can spend their fortunes on millions of uber-genius children if they please, and they don't have to spend it all on their contest to replace ourselves with the steel & copper successors.

7Ebenezer Dukakis8mo

Altman and Musk are arguably already misaligned relative to humanity's best interests. Why would you expect smarter versions of them to be more aligned? That only makes sense if we're in an "alignment by default" world for superbabies, which is far from obvious.

5lemonhope8mo

I would vote to be ruled by their carbon children instead of their silicon children for certain

8Ebenezer Dukakis8mo

There's a good chance their carbon children would have about the same attitude towards AI development as they do. So I suspect you'd end up ruled by their silicon grandchildren.

5lemonhope8mo

Good point! I didn't think that far ahead

[-]Pretentious Penguin8mo71

Nobel prize winners (especially those in math and sciences) tend to have IQs significantly above the population average.

There is no Nobel prize in math. And the word "especially" would imply that there exists data on the IQs of Nobel laureates in literature and peace which shows a weaker trend than the trend for sciences laureates; has anybody ever managed to convince a bunch of literature Nobel laureates to take IQ tests? I can't find anything by Googling, and I'm skeptical.

To be clear, the general claim that people who win prestigious STEM awards have above-average IQs is obviously true.

[-]Mo Putera8mo171

(To be clear: I agree with the rest of the OP, and with your last remark.)

has anybody ever managed to convince a bunch of literature Nobel laureates to take IQ tests? I can't find anything by Googling, and I'm skeptical.

I just read this piece by Erik Hoel which has this passage relevant to that one particular sentence you quoted from the OP:

Consider a book from the 1950s, The Making of a Scientist by psychologist and Harvard professor Anne Roe, in which she supposedly measured the IQ of Nobel Prize winners. The book is occasionally dug up and used as evidence that Nobel Prize winners have an extremely high IQ, like 160 plus. But it’s really an example of how many studies of genius are methodologically deeply flawed. ...
Roe never used an official IQ tests on her subjects, the Nobel Prize winners. Rather, she made up her test, simply a timed test that used SAT questions of the day. Why? Because most IQ tests have ceilings (you can only score like a 130 or 140 on them) and Roe thought—without any evidence or testing—that would be too low for the Nobel Prize winners. And while she got some help with this from the organization that created the SATs, she admits:
The test I used is not one

... (read more)

1Pretentious Penguin8mo

It should be noted that the psychologists and anthropologists in the above tables were not selected based on winning a Nobel prize, nor any prize. On pages 51-52 of The Making of a Scientist Roe writes and then lists a bunch of other professors involved in rating the list, and "the men who ranked at the top were selected, with some adjustment so as to include representatives of different sorts of psychology." (Incidentally, I wonder whether Professor Boring's lectures lived up to his name.)

[-]Metacelsus8mo70

Unfortunately monkeys (specifically marmosets) are not cheap. To demonstrate germline transmission (the first step towards demonstrating safety in humans), Sergiy needs $4 million.

And marmosets are actually the cheapest monkey. (Also, as New World monkeys, marmosets are more distantly related to humans than rhesus or cynomolgus monkeys are.)

5GeneSmith8mo

Do you have any estimate of how much more expensive testing in cynomolgus macaques or rhesus monkeys would be?

[-]Metacelsus8mo100

As a rough estimate, I think 3x to 5x more expensive. Marmosets are smaller (smaller than squirrels) whereas macaques (rhesus/cyno) are about 10x bigger (6 kg). And macaques take longer to develop (3 years vs. 18 months until adulthood). Finally, macaques are in high demand and low supply for pharma research.

But the benefit is that methods developed in macaques are more likely to translate to humans, due to the closer evolutionary relationship. Marmosets are a bit unusual in their embryonic development (two twin embryos share a common, fused placenta!)

[-]CRISPY8mo6-3

From a sales perspective, I find myself bewildered by the approach this article takes to ethics. Deriding ethical concerns then launching into a grassroots campaign for fringe primate research into genetic hygiene and human alignment is nonstarter for changing opinions.

This article, and another here about germ engineering, are written as if the concepts are new. The reality is that these are 19th century ideas and early attempts to implement them are the reason for the ethical concerns.

Using the standard analogical language of this site, AI and... (read more)

7TsviBT8mo

You seem like you might have read and thought about this a fair bit. Is that the case? Would you be up for a conversation about the questions you raise here (maybe that we record and possibly post)? Some of my thoughts are here https://www.lesswrong.com/posts/DfrSZaf3JC8vJdbZL/how-to-make-superbabies?commentId=ZeranH3yDBGWNxZ7h and here (just a list of things that might go wrong) https://berkeleygenomics.org/articles/Potential_perils_of_germline_genomic_engineering.html.

6kman8mo

The audience that needs to be convinced isn't the target audience of this post. But overall your point is taken.

[-]Donald Hobson8mo60

I'm not quite convinced by the big chicken argument. A much more convincing argument would be genetically selecting giraffes to be taller or cheetah to be faster.

That is, it's plausible evolution has already taken all the easy wins with human intelligence, in a way it hasn't with chicken size.

[-]kman8mo*100

If evolution has already taken all the easy wins, why do humans vary so much in intelligence in the first place? I don't think the answer is mutation-selection balance, since a good chunk of the variance is explained by additive effects from common SNPs. Further, if you look at the joint distribution over effect sizes and allele frequencies among SNPs, there isn't any clear skew towards rarer alleles being IQ-decreasing.

For example, see the plot below of minor allele frequency vs the effect size of the minor allele. (This is for Educational Attainment, a highly genetically correlated trait, rather than IQ, since the EA GWAS is way larger and has way more hits.)

If we do a Bayesian adjustment of the effect sizes for observation noise, assuming EA has a SNP heritability of 0.2 and 20,000 causal variants with normally distributed effects:

I've repeated this below for an IQ GWAS:

You can see that the effect sizes look roughly symmetrically distributed about zero, even for fairly rare SNPs with MAF < 1%.

3jbash8mo

Who says humans vary all that much in intelligence? Almost all humans are vastly smarter, in any of the ways humans traditionally measure "intelligence", than basically all animals. Any human who's not is in seriously pathological territory, very probably because of some single, identifiable cause. The difference between IQ 100 and IQ 160 isn't like the difference between even a chimp and a human... and chimps are already unusual. Eagles vary in flying speed, but they can all outfly you. Furthermore, eagles all share an architecture adapted to the particular kind of flying they tend to do. There's easily measurable variance among eagles, but there are limits to how far it can go. The eagle architecture flat out can't be extended to hypersonic flight, no matter how much gene selection you do on it. Not even if you're willing to make the sorts of tradeoffs you have to make to get battery chickens.

3kman8mo

So on one hand, I sort of agree with this. For example, I think people giving IQ tests to LLMs and trying to draw strong conclusions from that (e.g. about how far off we are from ASI) is pretty silly. Human minds share an architecture that LLMs don't share with us, and IQ tests measure differences along some dimension within the space of variation of that architecture, within our current cultural context. I think an actual ASI will have a mind that works quite differently and will quickly blow right past the IQ scale, similar to your example of eagles and hypersonic aircraft. On the other hand, humans just sort of do obviously vary a ton in abilities, in a way we care about, despite the above? Like, just look around? Read about Von Neumann? Get stuck for days trying to solve a really (subjectively) hard math problem, and then see how quickly someone a bit smarter was able to solve it? One might argue this doesn't matter if we can't feasibly find anyone capable of solving alignment inside the variation of the human architecture. But Yudkowsky, and several others, with awareness and understanding of the problem, exist; so why not see what happens if we push a bit further? I sort of have this sense that once you're able to understand a problem, you probably don't need to be that much smarter to solve it, if it's the sort of problem that's amenable to intelligence at all. On another note: I can imagine that, from the perspective of evolution in the ancestral environment, that maybe human intelligence variation appeared "small", in that it didn't cache out in much fitness advantage; and it's just in the modern environment that IQ ends up conferring massive advantages in ability to think abstractly or something, which actually does cache out in stuff we care about.

3Walkabout8mo

I've been over a big educational attainment GWAS, and one of the main problems with them seems to me to be that they make you think that the amount of schooling a human gets is somehow a function of their personal biochemistry. If you really want to look at this, you need to model social effects like availability, quality, and affordability of education, the different mind shapes needed to do well in school for people who are oppressed to different degrees or in different ways, whether people have access to education modalities or techniques shaped to fit their mind, whether the kid is super tall and gets distracted from grad school by a promising career in professional basketball, whether or not their mental illnesses are given proper care, and so on. If you measure how many years of education are afforded to a random human you mostly get social factors. If you're looking at the same big EA GWAS that threw out all non-Europeans that I'm thinking of, they didn't look at any of that. I don't believe a sufficient model is common practice, because as noted in the thread there is effectively no applied branch of the field that would expose the insufficiency of the common models.

2tantrev7mo

@kman - can you explain more about how you did the Bayesian adjustment you did?

2Donald Hobson8mo

That is good evidence that we aren't in a mutation selection balance. There are also game theoretic balances. Here is a hypothesis that fits my limited knowledge of genetics, and is consistent with the data as I understand it and implies no huge designer baby gains. It's a bit of a worst plausible case hypothesis. But suppose we were in a mutation selection balance, and then there was an environmental distribution shift. The surrounding nutrition and information environment has changed significantly between the environment of evolutionary adaptiveness, and today. A large fraction of what was important in the ancestral world was probably quite emotion based. Eg calming down other tribe members. Winning friends and influencing people. In the modern world, abstract logic and maths are somewhat more important than they were, although the emotional stuff still matters too. Iq tests mostly test the more abstract logical stuff. Now suppose that the optimum genes aren't that different compared to ambient genetic variation. Say 3 standard deviations.

4GeneSmith8mo

It's just very hard for me to believe there aren't huge gains possible from genetic engineering. It goes against everything we've seen from a millenia of animal breeding. It goes against the estimates we have for the fraction of variance that's linear for all these highly polygenic traits. It goes against data we've seen from statisitcal outliers like Shawn Bradley, who shows up as a 4.6 standard deviation outlier in graphs of height: Do I buy that things will get noisier around the tails, and that we might not be able to push very far outside the +5 SD mark or so? Sure. That seems unlikely, but plausible. But the idea that you're only going to be able to push traits by 2-3 standard deviations with gene editing before your predictor breaks down seems quite unlikely. Maybe you've seen some evidence I haven't in which case I would like to know why I should be more skeptical. But I haven't seen such evidence so far.

2kman8mo

I'm sort of confused by the image you posted? Von Neumann existed, and there are plenty of very smart people well beyond the "Nerdy programmer" range. But I think I agree with your overall point about IQ being under stabilizing selection in the ancestral environment. If there was directional selection, it would need to have been weak or inconsistent; otherwise I'd expect the genetic low hanging fruit we see to have been exhausted already. Not in the sense of all current IQ-increasing alleles being selected to fixation, but in the sense of the tradeoffs becoming much more obvious than they appear to us currently. I can't tell what the tradeoffs even were: apparently IQ isn't associated with the average energy consumption of the brain? The limitation of birth canal width isn't a good explanation either since IQ apparently also isn't associated with head size at birth (and adult brain size only explains ~10% of the variance in IQ).

3Donald Hobson8mo

Yes. I expect extreme cases of human intelligence to come from a combination of fairly good genes, and a lot of environmental and developmental luck. Ie if you took 1000 clones of Von Neumann, you still probably wouldn't get that lucky again. (Although it depends on the level of education too) Some ideas about what the tradeoffs might be. Emotional social getting on with people vs logic puzzle solving IQ. Engineer parents are apparently more likely to have autistic children. This looks like a tradeoff to me. To many "high IQ" genes and you risk autism. How many angels can dance on the head of a pin. In the modern world, we have complicated elaborate theoretical structures that are actually correct and useful. In the pre-modern world, the sort of mind that now obsesses about quantum mechanics would be obsessing about angels dancing on pinheads or other equally useless stuff.

1kman8mo

Not sure I buy this, since IQ is usually found to positively correlate with purported measures of "emotional intelligence" (at least when any sort of ability (e.g. recognizing emotions) is tested; the correlation seems to go away when the test is pure self reporting, as in a personality test). EDIT: the correlation even with ability-based measures seems to be less than I expected. Also, smarter people seem (on average) better at managing interpersonal issues in my experience (anecdotal, I don't have a reference). But maybe this isn't what you mean by "emotional social getting on with people". There could have been a thing where being too far from the average caused interpersonal issues, but very few people would have been far from the average, so I wouldn't expect this to have prevented selection if IQ helped on the margin. Seems somewhat plausible. I don't think that specific example is good since engineers are stereotyped as aspies in the first place; I'd bet engineering selects for something else in addition to IQ that increases autism risk (systematizing quotient, or something). I have heard of there being a population level correlation between parental IQ and autism risk in the offspring, though I wonder how much this just routes through paternal age, which has a massive effect on autism risk. This study found a relationship after controlling for paternal age (~30% risk increase when father's IQ > 126), though the IQ test they used had a "technical comprehension" section, which seems unusual for an IQ test (?), and which seems to have driven most of the association. So I think there's two possibilities here to keep distinct. (1) is that ability to think abstractly wasn't very useful (and thus wasn't selected for) in the ancestral environment. (2) Is that it was actively detrimental to fitness, at least above some point. E.g. because smarter people found more interesting things to do than reproduce, or because they cared about the quality of life of their

[-]Knight Lee8mo*60

The problem is with society, politics and ethics, ~~rather than a technical problem.~~ in addition to a technical problem.

I think the solution should be to vividly demonstrate how effective and safe it is with animal studies, so that a lot of normal people will want to do it, and feel that not doing it is scarier than doing it.

If a lot of normal people want it, they will be able to get it one way or another (flying to another country, etc.).

3GeneSmith8mo

No, the problem really is technical right now. There may be additional societal and political problems afterwards. But none of those problems actually matter unless the technology works. Obviously we are going to do it in animals first. We have in fact DONE gene editing in animals many times (especially mice, but also some minor stuff in cows and other livestock). But you're correct that we need to test massive multiplex editing. My hope is we can have good data on this in cows in the next 1-3 years.

8Knight Lee8mo

Oops, sorry about saying it's not a technical problem. I should have read the post before replying. I have a bad habit. PS: my comment was about effectiveness demonstrating with animals not just safety testing with animals. If you have a mouse clearly smarter, healthier, etc. than the other mice it would leave a strong impression on people.

1cozyfae8mo

What do you think of the argument that "There may be additional technical problems afterwards. But none of those problems actually matter unless we have answers for societal and political problems."?

[-]saulius8mo61

I've just started reading and this seems very interesting and important. However, I find the discussion about embryos and scaling odd. I mean sentences like "If we had 500 embryos". Here is some quick info for women under 35, generated by ChatGPT:

A single egg collection usually retrieves 8-14 eggs. Out of those, only 4-6 embryos typically develop far enough to be tested, and about 50-60% of those will be genetically normal. This means that in most cases, only 2-4 embryos per cycle are actually viable for implantation.
Even in the best-case scenario, only ab

... (read more)

7kave8mo

There are various technologies that might let you make many more egg cells than are possible to retrieve from an IVF cycle. For example, you might be able to mature oocytes from an ovarian biopsy, or you might be able to turn skin cells into eggs.

2JenniferRM8mo

Wait, what? I know Aldous Huxley is famous for writing a scifi novel in 1931 titled "Don't Build A Method For Simulating Ovary Tissue Outside The Body To Harvest Eggs And Grow Clone Workers On Demand In Jars" but I thought that his warning had been taken very very seriously. Are you telling me that science has stopped refusing to do this, and there is now a protocol published somewhere outlining "A Method For Simulating Ovary Tissue Outside The Body To Harvest Eggs"???

5GeneSmith8mo

A brief summary of the current state of the "making eggs from stem cells" field: * We've done it in mice * We have done parts of it in humans, but not all of it * The main demand for eggs is from women who want to have kids but can't produce them naturally (usually because they're too old but sometimes because they have a medical issue). Nobody is taking the warning to not "Build A Method For Simulating Ovary Tissue Outside The Body To Harvest Eggs And Grow Clone Workers On Demand In Jars" because no one is planning on doing that. Even if you could make eggs from stem cells and you wanted to make "clone workers", it wouldn't work because every egg (even those from the same woman) has different DNA. They wouldn't even be clones.

3JenniferRM8mo

Oh huh. I was treating the "and make them twins" part as relatively easier, and not worthy of mention... Did no one ever follow up on the Hall-Stillman work from the 1990s? Or did it turn out to be hype, or what? (I just checked, and they don't even seem to be mentioned on the wiki for the zona pellucida.)

4TsviBT8mo

Look up "in vitro maturation". E.g. https://www.sciencedirect.com/science/article/pii/S0015028212017876 . I haven't evaluated this literature much, so I don't know exactly what can and can't be done. See maybe this review (not super clearly written). https://tjoddergisi.org/articles/doi/tjod.23911

2kave8mo

I'm not aware of a currently published protocol; sorry for confusing phrasing!

4GeneSmith8mo

Yes you're right. With current technology there's no way you could get anywhere close to 500 embryos. I know a couple trying to get 100 and even that seems crazy to me. 5-20 is more realistic for most people (and 5 is actually quite good if you have fertility issues). But we wanted to show 500 edits to compare scaling of gene editing and embryo selection and there wasn't any easy way to do that without extending the graph for embryo selection.

3saulius8mo

Thanks for clarifying. If you ever pitch your ideas to potential investors or something, I recommend avoiding talking about hundreds of embryos, or at least acknowledging that this is unrealistic with current technologies before doing so. When reading, I was a bit worried that you might be divorced from reality, thinking in sci-fi terms, not knowing the basic realities about IVF. This made it difficult for me to trust other things you were saying about domains I know nothing about. Just letting you know in case it's helpful :)

[-]Ya Polkovnik2mo50

I'm preemptively sorry if this question have already been raised, I don't feel like reading all the comments.

What am I doing now is reciting the points from Habr (a mostly-coding-oriented site where this article have been translated in Russian and criticized in comments)

The main criticism is against the key point of the article, the graphs of "edit N genes, get M bonus to IQ".

The arguments are:

There are only a few hundred IQ-related genes, and they're found through a correlation in over 240k people, so it's not necessary that you can just

... (read more)

5GeneSmith2mo

It's fine, I wouldn't expect you to read all the comment. There have been quite a few more genetic variants linked to IQ since this study was published in 2018. Herasight has the best IQ predictor I know of, which can explain over twice as much variance as the best predictors we had in 2018. Of course it's correct that in most cases we aren't highly certain which one of a cluster of variants is actually causing the effect we observe. But we don't need to know. We just need reasonably good odds. Then we can edit variants with the highest combined probability of causality * effect size. That's how we came up with the graph in the post: we didn't falsely assume we know which of the variants are causal. Yes, that's true if you're doing single shot editing, which is one of the limitations for the kind of germline engineering we're currently pursuing. But the more advanced editing protocols don't rely on single shot editing. They use iterated CRISPR, as explained in the post, and that in turn uses multiple rounds of editing. In each round, you're selecting cells that not only received many edits, but didn't contain any deal-breaker mutations. So at least in theory, there is no reason why adding more edits would break essential genomic functions. In practice, there are a lot of headaches here: base editors result in bystander edits, prime editors result in indels, and each cell division carries a certain risk of copying errors and chromosomal abnormalities. None of these are insurmountable, but overcoming them will require a decent bit of engineering. Do you have a link to the Habr article? I'm curious to read what people are saying.

1Ya Polkovnik1mo

Thanks for replying! The link is: https://habr.com/ru/articles/933324/

[-]Mars_Will_Be_Ours8mo50

Thank you for writing this article! It was extremely informative and I am very pleased to learn about super-SOX. I have been looking for a process which can turn somatic cells into embryonic stem cells due to unusual personal reasons, so by uncovering this technology you have done me a great service. Additionally, I agree that pursing biological superintelligence is a better strategy than pursuing artificial superintelligence. People inherit some of their moral values from their parents, so a superintelligent human has a reasonable probability of being a g... (read more)

5GeneSmith8mo

I'm glad you liked the article! Brain size is correlated with intelligence at maybe 0.3-0.4. If you were to just brain size max I think it would probably not yield the outcomes you actually want. It's better to optimize as directly as you can for the outcome you want.

2Mars_Will_Be_Ours8mo

Good point. I am inherently drawn to the idea of increasing brain size because I favor extremely simple solutions whenever possible. However, a more focused push towards increasing intelligence will produce better results as long as the metric used for measuring intelligence is reliable. I still think that increasing brain size will take a long time to reach diminishing returns due to its simplicity. Keeping all other properties of a brain equal, a larger brain should be more intelligent. There is also one other wildly illegal approach which may be viable if you focus on increasing brain size. You might be able to turn a person, perhaps even yourself, into a biological superintelligence. By removing much of a person's skull and immersing the exposed brain in synthetic cerebrospinal fluid, it would be possible to restart brain growth in an adult. You could theoretically increase a person's brain size up to the point where it becomes difficult to sustain via biological or artificial means. With their physical abilities crippled, the victim must be connected to robot bodies and sense organs to interact with the world. I don't recommend this approach and would only subject myself to it if humanity is in a dire situation and I have no other way of gaining the power necessary to extract humanity from it.

2lemonhope8mo

Women/girls with big heads tend to hit their heads but you can solve that with bigger arms.

[-]nabokos3mo4-7

Are we still talking about human beings – or optimization projects? You can’t predict or engineer how a baby will turn out. What troubles me most is how little attention is paid to emotional attachment, which is arguably the cornerstone of healthy development. This reads more like a plan for growing babies in vitro than raising actual children. Honest question: do you have kids?

Also, much of the terminology you use feels superficial or misapplied. Science and education aren’t just about memorizing buzzwords – they require deep understanding, and that takes... (read more)

[-]GeneSmith3mo166

Hi Nabokos,

I appreciate the comment. I think many academically inclined follks probably have similar views to yours. Let me explain my thinking here:

What troubles me most is how little attention is paid to emotional attachment, which is arguably the cornerstone of healthy development. This reads more like a plan for growing babies in vitro than raising actual children.

If I were to go into the ins and outs of emotional attachment, this already long post would have been at least twice the length. And seeing as I am not an expert in the area, I hardly think it would have been useful to the average reader.

Of course emotional attachment is important. It's one of the most important things for happy, healthy childhood development.

But there are many good books on that topic and I don't think everyone who writes about any aspect of childhood or babies needs to include a section on the topic. If you think there are good resources people here should read, please post them!

You can’t predict or engineer how a baby will turn out.

It's certainly true you can't predict EXACTLY how a baby will turn out, but you CAN influence predispositions. In fact, most of parenting is about exactly this! How to c... (read more)

4nabokos3mo

Thanks for the detailed response. I appreciate your willingness to engage in dialogue. To clarify: I didn’t mean to imply that you literally stated hypertension is caused by a single gene – rather, I was pointing to a broader pattern I noticed in your post, where complex traits seemed to be discussed in a somewhat reductionist way. That might not have been your intent, but that’s how it came across to me as a physician who deals with multifactorial conditions daily. I also understand that emotional attachment wasn’t your focus – fair enough. Still, I think omitting it entirely risks presenting a view of childhood development that’s skewed towards instrumental or mechanistic thinking. For people unfamiliar with the developmental literature, that can unintentionally reinforce the idea that kids are optimization projects rather than human beings growing in relational contexts. If that wasn’t your goal, I’d just gently suggest flagging this limitation more clearly. On the topic of predictability and engineering – sure, we can influence predispositions, but the point I was trying to make is epistemological: the level of uncertainty and interdependence in human development makes the engineering metaphor fragile. Medicine, to your point, does aim to “figure out” complex systems – but it’s also deeply aware of its limitations, unintended consequences, and historical hubris. That humility didn’t come through strongly in your piece, at least to me. You clearly put a lot of thought into the genetic correlations and tradeoffs, and I don’t doubt your reasoning there. But I do wonder: do we really know enough about gene–environment interactions to be confident in the long-term effects of shifting polygenic profiles at scale? Especially given how much of our current “environment” is in flux, including things like diet, social structures, and digital exposure? These aren’t rhetorical questions – just areas where I think caution is warranted. Again, I don’t mean this as a taked

[-]GeneSmith3mo184

On the topic of predictability and engineering – sure, we can influence predispositions, but the point I was trying to make is epistemological: the level of uncertainty and interdependence in human development makes the engineering metaphor fragile. Medicine, to your point, does aim to “figure out” complex systems – but it’s also deeply aware of its limitations, unintended consequences, and historical hubris. That humility didn’t come through strongly in your piece, at least to me.

Perhaps so. But the default assumption, seemingly made by just about everyone, is that there is nothing we can do about any of this stuff.

And that's just wrong. The human genome is not a hopeless complex web of entangled interactions. Most of the variance in common traits is linear in nature, meaning we can come up with reasonably accurate predictions of traits by simply adding up the effects of all the genes involved. And thus by extension, if we could flip enough of these genes, we could actually change people's genetic predisposition.

Furthermore, nature has given us the best dataset ever in genetics, which is billions of siblings that act as literal randomized control trials for the effect of genes ... (read more)

[-]lemonhope7mo40

23andMe (and all their data) seems to be for sale at a cheap discount.

4GeneSmith7mo

I’ve checked. Have heard from multiple people they “it’s not for sale in reality” I don’t have any details yet. But obviously am interested.

[-]gustavodnm7mo40

This is my first comment in this forum. I discovered you guys early this morning and read the posts until about 5 a.m., completely hypnotized. Considering the current context of social media, it’s been a long time since a deep read has held my attention like this. Man, this has been the most interesting post I’ve accessed so far. I feel that being part of this community and connecting with people like you will add a lot to my self-development. Congratulations on the text.

Enviado de SP BR, texto original em portugues traduzido com o uso de IA.

[-]Evan R. Murphy8mo40

Is there a summary of this post?

2GeneSmith8mo

https://x.com/GeneSmi96946389/status/1892721828625264928

[-]AlexanderTheGreat13378mo41

Interesting article! Thanks for writing. But I am suspicious of claims about increasing IQ because, if it was possible, surely someone would have done a proof-of-concept with animals.

It's difficult to do such experiments with humans because (1) ethical issues; (2) humans take 9 months to be born (long gestation period); and (3) a few years for them to mature enough to get any objective results. I imagine the effort to increase IQ would be iterative. That needs a quick feedback loop which is not possible with humans.

There must be some animal with (1) no eth... (read more)

3GeneSmith8mo

First of all, no one has really done large scale genetic engineering of animals before, so we wouldn't know. Almost all mouse studies or genetic studies in other animals are very simple knockout experiments where they break a protein to try to assess its function. We really haven't seen a lot of multiplex editing experiments in animals yet. But even if someone were to do that it would be hard to evaluate the effects on intelligence in animals. The genetic variants that control IQ in humans don't always have analogous sequences in animals. So you'd be working with a subset of possible edits at best. The first proof of concept here will probably be something like "do tons of edits in cows to make them produce more milk and beef". In fact, that's one of the earliest commercial applications of this multiplex editing tech. We're hoping to show a demonstration of this in the next couple of years as one of the first steps towards demonstrating plausible safety and efficacy in humans.

[-]ChristianKl8mo4-1

Currently, we have smart people who are using their intelligence mainly to push capabilities. If we want to grow superbabies into humans that aren't just using their intelligence to push capabilities, it would be worth looking at which kind of personality traits might select for actually working on alignment in a productive fashion.

This might be about selecting genes that don't correlate with psychopathy but there's a potential that we can do much better than just not raising psychopaths. If you want to this project for the sake of AI safety, it would be crucial to look into what kind of personality that needs and what kind of genes are associated with that personality.

9GeneSmith8mo

I think we need to think more broadly than this. There's some set of human traits, which is a combination of the following: * Able to distinguish prosocial from antisocial things * Willing and able to take abstract ideas seriously * Long term planning ability * Desire to do good for their fellow humans (and perhaps just life more broadly) Like, I'm essentially trying to describe the components of "is reliably drawn towards doing things that improve the lives of others". I don't think there's much research on it in the literature. I haven't seen a single article discuss what I'm referring to. It's not exactly altruism, at least not the naive kind. You want people that punish antisocial behavior to make society less vulnerable to exploitation. Whatever this thing is, this is one of the main things that, at scale, would make the world a much, much better place.

2ChristianKl8mo

Do you have hope that someone else does the required research, so that it's ready by the time the first superbabies are created? If not, do you think it's okay to create superintelligent babies without it?

2GeneSmith8mo

I think superbabies would still have a massive positive impact on the world even if all we do is decrease disease risk and improve intelligence. But with this kind of thing I think the impact could be very robustly positive to an almost ridiculous degree. My hope is as we scale operations and do more fundraising we can fund this kind of research.

[-]Kris Moore8mo4-2

It is becoming increasingly clear that for many traits, the genetic effect sizes estimated by genetic association studies are substantially inflated for a few reasons. These include confounding due to uncontrolled population stratification, such as dynastic effects, and perhaps also genetic nurture^[1]. It is also clear that traits strongly mediated through society and behaviour, such as cognitive ability, are especially strongly affected by these mechanisms.

You can avoid much of this confounding by performing GWAS on only the differences between sibl... (read more)

4kman8mo

We accounted for inflation of effect sizes due to assortative mating, assuming a mate IQ correlation of 0.4 and total additive heritability of 0.7 for IQ. IIUC that R = 0.55 number was just the raw correlation between the beta values of the sibling and population GWASes, which is going to be very noisy given the small sample sizes and given that effects are sparse. You can see that the LDSC based estimate is nearly 1, suggesting ~null indirect effects.

4kman8mo

Subtle population stratification not accounted for by the original GWAS could still be an issue, though I don't expect this would inflate the effects very much. If we had access to raw data we could take into account small correlations between distant variants during finemapping, which would automatically handle assortative mating and stratification.

2Kris Moore8mo

I think you should take seriously that in the first paper linked in my comment, the population-wide SNP heritability for cognitive ability is estimated at 0.24 and the within-sibship heritability at 0.14. This is very far from the 0.7 estimate from twin studies. While a perfect estimate of direct additive heritability would be higher than 0.14, I don't think that rare variants (and gene-gene interactions, but this would no longer be additive heritability) would get you anywhere close to 0.7. Note also that UK Biobank with its purportedly poor IQ test represents only ~30% of the sample size in that paper. Instead, I think it is becoming clear that traditional twin studies made overly strong assumptions about shared and non-shared environments, such that they over-estimated the contribution of genetics to all kinds of traits from height to blood creatinine concentration (compare gold-standard RDR estimates vs twin estimates here). As implied in my original comment, this is likely especially true for traits strongly mediated by society and behaviour. I find it somewhat counter-intuitive, but this kind of finding keeps cropping up again and again in papers that estimate direct heritability with the most current methods.

2kman8mo

I'll need to do a deep dive to understand the methods of the first paper, but isn't this contradicted by the recent Tan et. al. paper you linked finding SNP heritability of 0.19 for both direct and population effects of intelligence (which matches Savage Jansen 2018)? They also found ~perfect LDSC correlation between direct and population effects, which would imply the direct and population SNP heritabilities are tagging the exact same genetic effects. (Also interesting that 0.19 is the exactly in the middle of 0.24 and 0.14, not sure what to make of that if anything).

2kman8mo

Actually I don't think this is correct, it accounted for sampling error somehow. I'll need to look into this deeper.

[-]Probilio5mo30

Hello,

I'm a Maternal-Fetal Medicine Specialist, a "High Risk" OB/GYN. This is a very interesting post, but firmly set in the realm of bench work. There's a reason Biotech has historically over-promised and underperformed. Lab results do not correlate well with clinical results in the human model. Human fertility and development are more complicated and vastly less understood than in most any other mammalian model. Human development and pregnancy are some of the least studied areas in medicine. No one in any democratic na... (read more)

[-]Michael Harrop7mo*3-3

I agree with the majority of this, especially the part where we are in a race to increase our own intelligence before we destroy ourselves with AI or something else.

But I think there's something missing in your analysis.

"Genes are the piano; the microbiome is the pianist."

It’s time to admit that genes are not the blueprint for life (Feb 2024) https://www.nature.com/articles/d41586-024-00327-x - I see it's closed access now; here are some quotes.

More on the microbiome & genetics: https://humanmicrobiome.info/genetics/

Examples of how important the gut mi... (read more)

7gwern7mo

The human microbiome is irrelevant to this topic. The microbiome is highly heritable (usual twin studies & SNP heritabilities), and it is caused by genes and the environment, as well as unstable; its direct causal effects in normal humans are minimal. We know that it is supremely irrelevant because environmental changes like antibiotics or new food or global travel which produce large changes in personal (and offspring) microbiomes do not produce large changes in intelligence (of oneself or offspring); and most dramatically, germ-free humans exist and are of normal or even above-average intelligence, eg the fascinating mistakes and delusions of David despite his high intelligence. (Amusingly, germ-free mice apparently even live longer.) Microbiome research is, in general, very low quality and can't be taken seriously - look at your link: Most of this page is meaningless mouse studies (infamous for not replicating and getting whatever result the experimenter wants and the animal model literature having huge systemic biases), and the handful of actual human studies I see here are all garbage - things like cross-sectional studies with large known familial confounding, or heavy reliance on things like breastfeeding where the beneficial effects disappear when controlling for just some confounds. This also goes for much-touted correlations like autism. There's not a single result on this page that provides a shred of evidence for your implied thesis that microbiome interventions could, even in theory, possibly matter to 'how to make superbabies'. It doesn't. He was right. BTW, you remember what happened in 2021, right? EDIT: If anyone cares, I'm not bothering to respond to Harrop's comment in depth because I think his Gish-gallopy response manages to exemplify many of the criticisms I already made, where they are not outright non-responsive (eg. his 'disagreement' about the reasons why germ-free mice live longer is obviously irrelevant to my point that they do), and I'

1Olli Savolainen7mo

I agree that woo is bad. And microbiome is of course irrelevant wrt boosting IQ. But a good part of the post was about improving health, and microbes do have serious downsides on that front. If you don't have the good ones you are at a much greater risk of being colonized by the bad ones. And disease still has a non-zero negative effect on people's brain development and cognition. Removing bad behaviour from microbiome would be quite a bit more effective and easier than fixing genes, for fighting disease. And many of the diseases with a significant genomic risk scores mentioned in the post probably have an unknown necessary pathogenic cause. Here's a paper (Cochran&Ewald) with simple powerful arguments, I always try to push it to any doctors I meet.

0Michael Harrop7mo

Oh my god, what a disturbingly overconfident & erroneous comment. Especially coming from someone who has been immersed in science for so many years. I recognize your name from Reddit from over 10 years ago. Due to Brandolini's law, your comment made me look up how to block users on Lesswrong, which apparently isn't possible. I now have to waste a huge amount of time debunking your egregious misinformation. I will only do it this once because in my experience, people who exhibit this kind of behavior will continue it. So in the future I will simply refer to this exchange as evidence that you are not someone who deserves to be taken seriously or responded to. On any evidence-based website, your comment is the type that deserves a warning and then a permanent ban if it happens again. That you've had an active account on this website for 15 years makes me want to avoid this website. The topic is how to make people/babies healthier, better developed, and more intelligent. Anyone who reviews this information should be able to conclude that your statement is ridiculously false: * https://humanmicrobiome.info/maternity/ * https://humanmicrobiome.info/brain/ * https://humanmicrobiome.info/aging/ * Many more https://humanmicrobiome.info/intro/ You started off with largely irrelevant statements and ended with severe misinformation. FMT (fecal microbiota transplant) studies demonstrate causation. You can look through the humanmicrobiome.info wiki or do a literature search to see how many FMT studies there are showing "non-minimal" effects. So much so that a 2020 review said they thought the results were implausible. Some examples are the plethora of studies showing that the benefits of fasting, the ketogenic diet, and other dietary interventions, are dependent on the gut microbiome, and the benefits can be transferred via FMT. And the same goes for exercise, grip strength, and muscle mass. Firstly, this is false. * Low-dose penicillin in early life induces long-te

[-]wonder7mo32

Maybe I missed this in the article itself - are there plans to make sure the superbabies are aligned and will not abuse/overpower the non-engineered peers?

4GeneSmith7mo

I certainly hope we can do this one day. The biobanks that gather data used to make the predictors we used to identify variants for editing don't really focus on much besides disease. As a result, our predictors for personality and interpersonal behavior aren't yet very good. I think as the popularity of embryo selection continues to increase, this kind of data will be gathered in exponentially increasing volumes, at which point we could start to think about editing or selecting for the kinds of traits you're describing. There will be an additional question to what degree parents will decide to edit for those traits. We're going to have a limited budget for editing and for selection for quite some time, so parents will have to choose to make their child kinder and more benificent to others at the expense of some other traits. The polygenicity of those personality traits and the effect sizes of the common alleles could have a very strong effect on parental choices; if you're only giving up a tiny bit to make your child kinder then I think most parents will go for it. If it's a big sacrifice because it requires like 100 edits, I think far fewer will do so. It may be that benificence towards others will make these kinds of children easier to raise as well, which I think many parents would be interested in.

[-]ConnorH8mo30

Very interesting, has the vibe of technology that's at least 20 years out from really hitting (assuming ASI is later than that) (though even then maybe progress speeds up? Idk, a lot of the barriers will be regulatory and certain parts won't be able to move faster than human growth and development).

The biggest surprise to me was probably that IQ selection was already happening without much fanfare - I had thought that there would be some law on the books, especially given the culture wars around pro choice/life and current admin. Perhaps it skates by because it's rare, expensive, inconvenient, screening rather than editing, and a low boost to begin with?

[-]jr8mo30

We’ve increased the weight of chickens by about 40 standard deviations relative to their wild ancestors, the red junglefowl. That’s the equivalent of making a human being that is 14 feet tall

I realize this is a very trivial matter on a very interesting post, and I don't like making a habit of nitpicking. But this feels interesting for some reason. Perhaps it's just because of the disturbing chicken visuals, I don't know.

To my credit, I actually made an effort to figure out the author reached their conclusion, and I believe I did. The average ad... (read more)

[-]Kongo Landwalker8mo30

I had a couple-year long obsession with evolution simulators. And that hobby convinced me, that longevity is a pretty risky thing. Most of dominant species in the simulations went extinct not because of new species emerging and conflicting, but because of longevity causing problems.

1) Reduces amount of resources available to the new generation. That increases competition, and this kind of competition is evolutionary force which encourages being a long-living line, because it is easier to accumulate experience and use it later. Creates a feedback loop... (read more)

3cousin_it8mo

Good point. But I think the real game changer will be self-modification tech, not longevity tech. In that case we won't have a "slow adaptation" problem, but we'll have a "fast adaptation in weird directions" problem which is probably worse.

[-]catubc8mo30

Great write up!

Why don't you do this in a mouse first? The whole cycle from birth to phenotype, including complex reasoning (e.g. bayesian inference, causality) can take 6 months.

4GeneSmith8mo

I would love to try this in mice. Unfortunately our genetic predictors for mice are terrible. The way mouse research works is not at all like how one would want it to work if we planned to actually use them as a testbed for the efficacy of genetic engineering. Mice are mostly clones. So we don't have the kind of massive GWAS datasets on which genes are doing what and how large the effect sizes are. Instead we have a few hundred studies mostly on the effects of gene knockouts to determine the function of particular proteins. But we're mostly not interested in knockouts for genetic engineering. 2/3rds of the disease related alleles in humans are purely single letter base pair changes. We have very little idea which specific single letter base pair changes affect things like disease risk in mice. MAYBE some of the human predictors translate. We haven't actually explicitly tested this yet. And there's at least SOME hope here; we know that (amazingly), educational attainment predictors actually predict trainability in dogs with non-zero efficacy. So perhaps there's some chance some of our genetic predictors for human diseases would translate at least somewhat to mice. We do need to do more thorough investigation of this but I'm not really that hopeful. I think a far better test bed is in livestock, especially cows. We have at least a few hundred thousand cow genomes sequenced and we have pretty well labelled phenotype data. It should be sufficient to get a pretty good idea of which alleles are causing changes in breed value, which is the main metric all the embryo selection programs are optimizing for.

4XFrequentist8mo

Dogs would be interesting - super smart working dogs might even have a viable labour market, and it seems like the evidence of supercanine IQ would be obvious in a way that's not true of any other species (just given how much exposure most people have to the range of normal canine intelligence). Sort of analogous to what Loyal is doing for longevity research.

1RichardJActon8mo

The lack of good population genetic information in animal models and deep phenotyping of complex behavioral traits is probably one of the biggest impediments to robust animal testing of this general approach.

3GeneSmith8mo

Well we have it in cows. Just not in mice.

2CronoDAS8mo

Do we have it in any other animal besides cows? Dogs? Housecats? Fruit flies? Guinea pigs? Any other short-lived animal commonly used in laboratory research that still has a decent amount of genetic diversity?

[-]jmh8mo3-4

I'm reminded of the old Star Trek episode with the super humans that were found in cryosleep that then took over the Enterprise.

While I do agree that this could be one potential counter to AI (unless the relative speed things overwhelm) but also see a similar type of risk from the engineered humans. In that view, the program needs to be something that is widely implemented (which would also make it potentially a x-risk case itself) or we could easily find ourselves having created a ruler class that views ordinary humans as subhuman on not deserving of full... (read more)

2CronoDAS8mo

Yeah, in Star Trek, genetic engineering for increased intelligence reliably produces arrogant bastards, but that's just so they don't have to show the consequences of genetic engineering on humans...

[-]Candace Hyde-Wang8mo32

What we need most is positive trait selection, not more IQ. Sociopathy is destroying our world and high IQ’s are not saving it. Get rid of the sociopaths and IQ may be salient.

[-]lemonhope8mo30

You could also make people grow up a bit faster. Some kids are more mature, bigger, etc than others at the same wall-clock age. If this doesn't conflict with lifespan then it would allow the superbabies to be productive sooner. Wouldn't want to rob someone of their childhood entirely, but 12 years of adolescence is long enough for lots of chase tag and wrestling.

9TsviBT8mo

This is a big ethical issue. Also, I haven't checked, but I'd guess that generally to have much of a noticeable effect, you're stepping somewhat to the edge of / outside of the natural range, which carries risks. Separately, this might not even be good on purely instrumental grounds; altriciality is quite plausibly really important for intelligence!

6lemonhope8mo

The wikipedia page picture has some evidence that you are right

[-]SmaWatters8mo20

Last year you guys wrote a post on adult intelligence enhancement. Does any of your research on super babies have implications for that, especially radical enhancement?

5GeneSmith8mo

There is some overlap with adult enhancement. Specifically, if we could make a large number of changes to the genome with a single transfection, that would be quite helpful.

[-]Anders Lindström8mo2-7

I know many of you dream of having an IQ of 300 to become the star researcher and avoid being replaced by AI next year. But have you ever considered whether nature has actually optimized humans for staring at equations on a screen? If most people don’t excel at this, does that really indicate a flaw that needs fixing?

Moreover, how do you know that a higher IQ would lead to a better life—for the individual or for society as a whole? Some of the highest-IQ individuals today are developing technologies that even they acknowledge carry Russian-roulette odds of wiping out humanity—yet they keep working on them. Should we really be striving for more high-IQ people, or is there something else we should prioritize?

[-]schme8mo20

Let's say it all turned out as expected and with minimal side effects we turned 100 IQ embryos into 150 IQ super-babies. Wouldn't this guarantee that every one of those enhanced children have zero chance at a normal upbringing? If they attended school at normal ages and progressed at a normal pace they would be horribly understimulated, but by attending accelerated programs they miss out on all the socialization and key moments we get by default. Eight-year-old high school graduates won't be found at prom or getting to have an awkward crush.

Now you might s... (read more)

3Rachel Reid8mo

I have also been thinking about this. It’s hard growing up being ‘odd’. Ideally, it would be great if the super babies grew up together, so that they would not feel isolated/weird. You don’t have to go full commune/kibbutz (although that could be fun), but I think the kids would benefit from knowing each other. This would not be so important for gen 1 of the super babies (+10iq from the parental mean means they would probably be fine in a regular gifted program), but I think gen 2 onwards will really need to lean on each other for support

1antisocial_spam@proton.me8mo

The worst argument in the world remains true: Someone's gonna do it. Might as well be someone who will make some attempt at doing it ethically and with kindness. You don't want this done by folks whose alignment is, say, chaotic evil.

[-]Ashin5mo10

Perhaps it will soon be the most Effective Altruism to raise your own superbabies. You're creating the next best thing to friendly AGI, in a context where the rest of the world is neglecting this low-hanging fruit. You could shape the first generation of smarter-than-unedited-human intelligence.

2GeneSmith5mo

Maybe we'll get there someday. I think for the next decade at least it's going to be hard to beat lead paint elimination or animal welfare initiatives that get a hundred million chickens out of battery cages.

[-]Mr Beastly8mo1-3

Interesting analysis, though personally, I am still not convinced that companies should be able to unilaterally (and irreversibly) change/update the human genome. But, it would be interesting to see this research continue in animals. E.g.

Provide evidence that they've made a "150 IQ" mouse or dog. What would a dog that's 50% smarter than the average dog behave like? or 500% smarter? Would a dog that's 10000% smarter than the average dog be able to learn, understand and "speak" in human languages?

Create 100s generations of these ... (read more)

[-]Mr Beastly8mo10

You can’t just threaten the life and livelihood of 8 billion people and not expect pushback.

"Can't" seems pretty strong here, as apparently you can... at least, so far...

Definitely "shouldn't" though...

[-]Min Liu8mo10

If genetic editing technology is first used by the wealthy, it may further exacerbate social inequality. This technology could be used to create a “super elite,” thereby further solidifying social stratification. The scenario of a “superbaby oligarchy” mentioned in the article is a worrying potential future.

4GeneSmith5mo

I agree this is a worry. Apart from this stuff just not mattering because AI takes over first, dramatic acceleration of inequality is my biggest worry. This tech almost certainly WILL accelerate inequality at the start. But in the long run I think there's no reason we can't make gene editing available for almost everyone. Editing reagents are cheap. We're working with at most a few microliters of editing agents (more realistically a few nanoliters). It costs a lot of money to collect the data and put it into biobanks, but once that is done you've got the data forever. And at SCALE the cost of absolutely everything comes down.

1JuliaHP5mo

I expect the main cost to be regulatory rather than technical, this seems to be a trend across various medicine. These costs might scale with the richest peoples ability to pay. examples-ish; - Needing expensive studies to get FDA (or other regulatory framework) approval, (and thus needing to sell at a premium to make up the loss). - Regulations which make market entry expensive (and favor the market leader by requiring bio-equivalence studies) which promote monopolies. - Need for expensive (time, money & training-capacity) general certifications for people to be allowed to administer narrow treatments. I don't have any domain-knowledge or analysis to illustrate my point, but I am curious to what extent you've (or someone else working on accelerating this technology) have thought about this. (metanote; I'm not meant to be discouraging of your work direction. to me it seems that both work on the tech itself, thinking about how to encourage favorable regulation and public opinion, as well as thinking about cultural downstream effects of the technology, are all extremely neglected.)

[-]ceba8mo*10

I like this. But I have questions. I hope for discussion, not answers.

Which people are most likely to get access to this tech first? is it likely that access will ever spread significantly beyond this group?

How does one raise a super child?

If they require different support from other children, whose resources are available to provide it?

Would there be some institution connected to the biotech company that would maintain a relationship with families or are they just selling a limited service?

What roles/responsibilities are the super ... (read more)

[-]Warp98mo10

@momom2 mentioned that there didn't seem to be anyone else in the comment section with a "kill it with fire" reaction. I don't think killing it with fire is the right approach either, but this research should be taken slow. I do not see a near future where superbaby injections aren't made prohibitively expensive, creating a new race of super-oligarchs. I think many of those hesitant researchers mentioned in the beginning of the article also see that future, which is probably why they aren't too excited to discuss those applications of their research.

[-]grrl_nrg8mo10

We don’t actually know that JK He accomplished what he said he did. He may have tried. But the results have not been independently verified. That would require genotyping the parents and children. So, premature to say that germline editing appears safe in humans.

2GeneSmith8mo

I'm not saying his experiments show germline editing is safe in humans. In fact He Jiankui's technique likely WASN'T safe. Based on some talks I heard from Dieter Egli at Colombia, He was likely deleting chromosomes in a lot of embryos, which is why (if I recall correctly) only 3 out of about ~30 embryos that were transferred resulted in live birth. Normally the live birth rate per transfer rate would be between 30 and 70%. It's also not entirely clear how effective the editing was because the technique He used likely created a fair degree of mosaicism since the editing continued after the first cell division. If the cells that ended up forming hematopoietic stem cells DIDN'T receive the edits then there would have been basically no benefit to the editing. Anyways, I'm not really trying to defend He Jiankui. I don't think his technique was very good nor do I think he chose a particularly compelling reason to edit (HIV transmission can be avoided with sperm washing or anti-retroviral drugs to about the same degree of efficacy as CCR5 knockout). I just think the reaction was even more insane. It doesn't make sense to ban germline editing just because one guy did it in a careless way. Yet in many places that's exactly what happened.

[-]TsviBT8mo218

You might not be tracking that there's a "unilateralist cloud" of behaviors. There's the norm of behavior, and then around that norm, there's a cloud of variation. The most extreme people (the unilateralist frontier) will do riskier things than the norm, and bad stuff will result. If a unilateralist doing unilateral stuff results in a deformed baby, or even of a legit prospective risk of a deformed baby (as you acknowledge existed in He Jiankui's experiments), that's bad. If society sees this happen, it means that their current norm is not conservative enough to keep the unilateralist frontier in the safe zone. So they adjust the norm to be more conservative.

When you write

This method has actually been used in human embryos before! In 2018 Chinese scientist He Jiankui created the first ever gene edited embryos by using this technique. All three of the children born from these embryos are healthy 6 years later (despite widespread outrage and condemnation at the time).

it sounds like you're implying that the outrage and condemnation were a claim that the experiment would actually have bad results. Which, surely to some extent they were. But also they were (legit, as you agree!) a c... (read more)

4GeneSmith8mo

Yes, I pretty much agree with this

[-]n s8mo1-4

This is a really interesting article and I commend the author for broaching the topic and proposing a way of getting around this stigmatization in the field. After diseases though i think an even greater priority than increasing IQ or lifespan is tackling what may seem to be a really trivial matter. And that is the issue of skin lightening. Though this may seem like a trivial topic I would say that conservatively tens of millions of mostly women possibly even hundreds of Millions in africa, asia, and the Middle East are literally poisoning and disfiguring ... (read more)

[-]gustavodnm7mo0-1

At one point in the text, it was mentioned that any ethical counterargument is not enough to demonize genome editing. Personally, I don’t see any issues with positive changes, but discussing this solely from the perspective of genetic science, without considering social science and all the people who already exist, is irresponsible. If the project receives the multimillion-dollar investment it needs and develops in the most efficient way possible, would the idea be to mass-produce gifted individuals until people born the traditional way are replaced? Maybe... (read more)

[-]Vasco Grilo8mo*0-3

Thanks for the post! I think genetic engineering for increasing IQ can indeed be super valuable, and is quite neglected in society. However, I would be very surprised if it was among the areas where additional investment generates the most welfare per $:

Open Philanthropy (OP) estimated that funding R&D (research and development) is 45 % as cost-effective as giving cash to people with 500 $/year.
People in extreme poverty have around 500 $/year, and unconditional cash transfer to them are like 1/3 as cost-effective as GiveWell's (GW's) top charities. GW

... (read more)

7kave8mo

I think your comment is supposed to be an outside view argument that tempers the gears-level argument in the post. Maybe we could think of it as providing a base-rate prior for the gears-level argument in the post. Is that roughly right? I'm not sure how much I buy into this kind of argument, but I also have some complaints by the outside views lights. I hope that is a roughly correct rendition of your argument. Here are my even-assuming-outside-view criticisms: 1. Even the Davidson model allows that the distribution for interventions that increase the rate/effectiveness of R&D (rather than just purchasing some at the same rate) could be much more effective. I think superresearchers (or even just a large increase in the number of top researchers) are such an intervention 2. To the extent we're allowing cause-hopping to enable large multipliers (which we must to think that there are potentially much more impactful opportunities than superbabies), I care about superbabies because of the cause of x-risk reduction! Which I think has much higher cost-effectiveness than growth-based welfare interventions.

1Vasco Grilo8mo

Thanks for the great summary, Kave! Nitpick. SWP received 1.82 M 2023-$ (= 1.47*10^6*1.24) during the year ended on 31 March 2024, which is 1.72*10^-8 (= 1.82*10^6/(106*10^12)) of the gross world product (GWP) in 2023, and OP estimated R&D has a benefit-to-cost ratio of 45. So I estimate SWP can only be up to 1.29 M (= 1/(1.72*10^-8)/45) times as cost-effective as R&D due to this increasing SWP’s funding. Fair points, although I do not see how they would be sufficiently strong to overcome the large baseline difference between SWP and general R&D. I do not think reducing the nearterm risk of human extinction is astronomically cost-effective, and I am sceptical of longterm effects.

[-]CasioTheSane8mo*00

While I think finding ways to make future generations healthier and smarter is a worthy goal, I don't think we understand enough yet to do this without potentially severe unintended consequences, and I wouldn't consider doing it myself with our current technology. It's a good bet that many of the seemingly deleterious mutations we'd like to eliminate also offer some benefit we don't understand- given that we have already discovered many instances of mutations with apparent intelligence/health tradeoffs, and disease resistance/health tradeoffs. If you're se... (read more)

[-]KatWoods8mo0-1

Very not important question: is Gene Smith your actual name or a pseudonymn?

Either way, it's the perfect name for the author of this post.

Hats off to you gene smith.

A blacksmith in a traditional forge, hammering a glowing strand of DNA on an anvil. Sparks fly as the DNA helix takes shape under the impact. The scene is illuminated by the fiery glow of the forge, with tools and metalwork surrounding the blacksmith. The blacksmith is muscular, wearing a leather apron, with intense focus on shaping the DNA strand.

7GeneSmith8mo

Ha, sadly it is a pseudonym. My parents were neither that lucky nor that prescient when it came to naming me.

6KatWoods8mo

Ah well. At least you can take credit for the name then.

[+]Amy Johnson8mo-9-2

LESSWRONG
LW

LESSWRONG
LW

629

How to Make Superbabies

629

629

How to make (slightly) superbabies

How to do better than embryo selection

Maximum human life expectancy

Is everything a tradeoff?

How to make an edited embryo

Embryos from stem cells

Iterated CRISPR

Sergiy Velychko and the story of Super-SOX

What is going on?

Super-SOX

Mice from stem cells

Why does super-SOX matter?

How do we do this in humans?

What if super-SOX doesn’t work?

Eggs from Stem Cells

Fluorescence-guided sperm selection

Embryo cloning

What if none of that works?

What about legal issues?

How we make this happen

Ahh yes, but what about AI?

There is currently no backup plan if we can’t solve alignment

Team Human

Appendix

Amendments to this article as of 2025-03-05

iPSCs were named after the iPod

On autoimmune risk variants and plagues

Two simples strategies for minimizing autoimmune risk and pandemic vulnerability

“I don’t want someone else’s genes in my child”

Could I use this technology to make a genetically enhanced clone of myself?

Why does super-SOX work?

How was the IQ grain graph generated?