This is a special post for quick takes by Samuel Ratnam. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
When evolutionary pressure is too high, you may get a population that is perfectly optimised for its current environment. Because of goodhart’s law, this means that the population is very vulnerable to a change in environment, such as a new virus, which may spread through the population and wipe it all out. Therefore a certain amount of slack/diversity within the population is adaptive in the face of Knightian uncertainty about future events.
What do you mean with evolutionary pressure being very high? What's a low/high evolutionary pressure environment?
we can build a toy model of this by assuming that organisms are 2-vectors, where each dimension ranges from 0-1, with some anti-correlation (for example, the organism is a unit vector). the intuition is like "birds that eat either seeds or nuts" where the components are "ease of eating nuts" and "ease of eating seeds", and beaks optimized for one are not capable at the other.
there's some game theory here, but we can sort of ignore it: we'll say that, each generation, there's some unit vector v representing the amount of seeds and nuts in the environment. a bird's fitness is like bird_vector . v, perhaps normalized to be between 0 and 1.
we can say that a bird "makes it" if its fitness is above some threshold p.
as p gets close to 1 (max fitness), or v is held constant for many generations, the bird population is winnowed until all birds are very near v. for ease of visualization, we can say that v was (1, 0). then all the birds will be like (.999, .001) or so.
if v jumps from (1, 0) one season to (0, 1) the next, all the birds may starve.
if p is lower, this is less likely to happen: the birds will be more like (.8, .2), and they'll be able to get some calories from the other food source.[1]
--
note: in a "real" evolutionary setting, population-level dynamics would come to bear. if such famine shocks were common, surviving bird populations would be ones that develop tendencies toward helping the less fortunate, or valuing diversity. if those memes are rapidly defeated by competition, the birds may evolve to have various personas within the species, whose populations are kept in check by intrasexual competition games, as in those lizards.
even if we raise p to the same level as in the previous experiment, the survivable window for v to land in will be wider.
note: in a "real" evolutionary setting, population-level dynamics would come to bear. if such famine shocks were common, surviving bird populations would be ones that develop tendencies toward helping the less fortunate, or valuing diversity.
That's sounds like "group evolution". As far as I understand most contemporary biologists don't think that a significant factor. It sounds to me like what you wrote is mostly speculation uninformed by engaging with the science.
hi! thanks for your reply.
first of all:
It sounds to me like what you wrote is mostly speculation uninformed by engaging with the science.
yes, spot on!
that said, i'm not sure that group selection is relevant here.
it's enough, in the toy model, to have some small fraction of the population perversely seek out "weirdos" as mates. this is explained selfishly: if you find yourself as a "weirdo", straightforward strategies are not likely to have a high enough mean to survive a few generations. but if you play for variance, you might get 'lucky' and see a famine next year. since a famine up-ends who is a weirdo and who is a normie, weirdos that get lucky with the timing will be highly successful in the next generation. if famines are frequent enough, this can keep perversion at a (perhaps small) positive rate in the population.
i would describe the above dynamic as "valuing diversity".
similarly with charity / "helping the less fortunate": tendency to insure your kin (give them a little extra, if they need it) is selected for by famines. in the model, the organism cannot tell the difference between "i am unusually fit, and next year this will continue" and "i am unusually fit, which means that i'm overtuned and will suffer next year". they can purchase some safety by donating.
of course, they should not donate so much that it affects their own viability. but there is some marginal calorie where it makes selfish sense to feed your brother. this allows the "help your brother" instinct to selfishly be carried by a wider population = a more robust population. the instinct itself will selfishly prefer a large number of less correlated bets.[1]
i don't have time to formalize these arguments, or build simulators. so it may be that my summary of the dynamics is wrong. but the dynamics are more subtle than "decline to have offspring as voodoo against malthus".
this is the main principle: in the face of uncertainty, genes should 'want' to make several uncorrelated bets, rather than "putting it all on red", so to speak. so genes which encode diversifying behaviors will outcompete cancerous ones, assuming enough uncertainty in the environment.
That's an interesting question in isolation. I guess the lowest selection would be when the whole tree is un-pruned, e.g. when bacteria split, but no bacteria die. But there would be still selection for speed of reproduction? Or, in opposite case, when you have only 1 bacterium, and it splits and you invariably kill one of its descendants, and repeat. That also has low selection I guess? So, something in between?
There are probably better answers if you know actual biology.
EDIT after consulting with some LLMs there is actually pretty standard terminology about this. Basically large heritable fitness differences and large population where variation can translate into frequency change.
https://en.wikipedia.org/wiki/Selection_coefficient
The link of selection coefficient goes to a concept that defined for a given genotype but for a population as a whole.
If you compare wild rats to lab rats you could say that there an abundance of resources for the lab rats. That does result in evolutionary pressure that differs from the pressure that exist in wild rats but it rewards mutation that result in more offspring per pregnancy and upregulating growth factors potentially at the cost of getting cancer later in life.
I think it's more like when the environment is perfectly stable for a very long time, so that even alleles which are only microscopically less fit get filtered out. If the environment keeps varying a little, there can be competing alleles, one of them more fit during one decade, another during another decade.
But there could be an equilibrium such that if one of the alleles gets less frequent it works as an advantage, and then the equilibrium would remain even after long time. This could happen e.g. with sexual selection -- when some visible trait becomes sufficiently rare, it may become a status symbol to get a mate with that rare trait.
Large connected landmasses have a lot more evolutionary pressure than Australia or small islands. This is demonstrated by the fact that they are inhabited by less advanced animals, which in turn is demonstrated by the fact that they are very vulnerable to invasive species from the large continents, while animals from the small landmasses mostly can't survive on the large continents. This is expected, because larger areas means there is room for more species which can compete with each other, and evolution generally faster in larger populations because it increases the probability that at least one organism has a beneficial mutation.
So according to your theory, we would expect species from large continents to be a lot more vulnerable to environmental changes than island species. This might be true, but it seems highly unlikely to me.
Alignment by induction
Assume model_0 is aligned (enough). What affordances can we give model_n to increase the probability that model_n+1 is (more) aligned? Model_n and model_n+1 might be over timesteps of a continual learning system, training steps or successor generations of models.To what extent do these methods also amplify misalignment? If a mistake is made along the chain, can we correct it using previous models? Maybe we can be nice to our models and work with them rather than constantly assuming an adversarial stance towards them.
Induction doesn't apply perfectly here, instead we'd like to know how well/badly conditioned our methods are. This is probably over and above the conditioning of the training process per se.
Condition numbers assume that errors propagate independently. Would be cool to build a coding theory for alignment so that we can plausibly correct random errors.
Are language models slowing the rate of linguistic evolution? It seems like adding a bunch of speakers of a language who cannot learn new words and regularly interact with a non-negligible proportion of world population ought to make our collective vocabulary stickier.
Do we have a good prior for reasoning about what neural networks converge to? It seems like neither the neither the solomonoff nor speed prior really take into account the computational constraints faced by neural networks. Do we have good reasons to expect these priors to tell us useful things about neural networks?
A computationally bounded agent can act as if they have more computational resources through externalising cognition into their environment. For example, we can use pen and paper to solve maths problems, and effectively simulate an agent that has a larger working memory. This is one reason why shard-theoretic agents may be adaptive. This is also one reason why mech interp is extremely hard in the limit.