A more correct but less concise statement of the analogy might be DL training : DL training code :: human learning : human genome, read as "DL training is to DL training code what human learning is to the human genome". This is sometimes contrasted with an alternative analogy DL training : DL-based AGI :: evolution : human mind.

Why the analogy is bad

Human learning mostly doesn't look much like DL training:

• Above the level of basic sensory stuff, humans have very little data to work with
  • And are correspondingly vastly more sample efficient at learning some sorts of things, such as theory of mind, language, even math
  • Contrast this to LLMs, which learn language and math through terabytes upon terabytes of examples (but currently do not manage to generalize as far as humans in their applications of these things)
• Human brains don't seem like terribly general learning machines below the level of deliberate thought (much less general than e.g. transformers)
  • A lot of cognitive abilities that vary between humans seem innate; they're readily picked up by those with the potential and remain out of reach to those without
    • E.g. perfect pitch, mental visualization (see aphantasia), etc.
  • To say nothing of the things no human can learn to do on a basic level
  • Brain organoids haven't managed to do anything more impressive than pong as far as I know

The higher level problem with this analogy is how it misleadingly/incorrectly assigns credit to processes for the work of building minds. To me, it's clear from the above points that human learning is doing very little of the work of building human minds; evolution did most of that work. On the other hand, the writing of DL training code (at least in the current paradigm) does very little of the work in building the mind of a hypothetical DL-based AGI: the DL process itself is doing most of the work.

What about the "DL training == evolution" analogy?

DL training also doesn't look much like biological evolution in some ways (e.g. much less of an information bottleneck). The reason this analogy works for the specific purpose of establishing an existence proof of inner misalignment with an outer optimizer is that it correctly identifies the optimization process which is doing the bulk of the actual mind-building work in each case.