If you teach an AI to fish, it might optimize its performance within a narrow scope. Teach it to teach itself to fish, and you've created a recursively self-improving AGI that is unaligned with human values by default and will most likely end up killing us all.

-- Eliezer, probably

It reminds me of what I told my kids about mathematicians being lazy and inventing ways of skipping work. For example, instead of repeated addition, find tricky ways to do multiplication. Or that fractions are just delayed divisions - we postpone doing the division as long as possible - including many multiplications on the way too (canceling down). Or think of summation rules.