If knowledge can create problems, it is not through ignorance that we can solve them.
—Isaac Asimov
Sabiola, the Homo Sapiens stone age lasted 300,000 years during which time we were the ultimate invasive species successfully covering the globe. They were ignorant about science but extremely knowledgeable about how to live a sustainable lifestyle.
We are knowledgeable about science but ignorant about how to live a sustainable lifestyle.
For those few who crawl out of the abyss of our own making, knowledge of how to live a sustainable lifestyle would be preferable to knowledge of science.
"I know not with which weapons WW3 will be fought, but WW4 will be fought with sticks and stones", Albert Einstein.
Why sticks and stones? Why not the left over weapons from the previous war, or at least scrap metal for swords, spear and arrow points? Einstein is implying is that if we are foolish enough to fight another world war the few survivors will have a powerful taboo against the remains of the scientific era. I would argue that Judkowski's crusade against AGI if taken to it's logical conclusion, reaches the same destination.
From the first farmers (our first science project) to AGI it is hard to see where we could have altered course, each development led to the next. Could we have stopped at the printing press, or the industrial revolution, or calculators?
Of course science is the best way to find the underlying source code of the universe, the advances in our common knowledge are astounding. And no sane person wants to see the end of humanity. Still, all our existential risks come from our World Wide Experiment With Science. How could we stop the the combination of humanity and science from creating existential risks to our survival?
Culture is formed from the important life lessons passed down through the generations and is far more important to our evolutionary success than genes. Our grand children's reproductive success depends more on the culture we gave our children than the genes. When AGI wipes us out it will be culture to blame, not genes. Culture can not keep up with the sweeping pace of change science has brought/
As a species we have enough weapons pointed right back at ourselves to extinguish humanity and our culture demands more, not less. So too with AGI. As a species we are incapable of formulating the 50 year plan that might save our way of life
Perhaps the most important life lesson that forms the culture of future generations is that humanity and science are incompatible.