This is a linkpost for https://scholars-stage.blogspot.com/2018/08/tradition-is-smarter-than-you-are.html
This is an excellent blog, and I am really pleased you introduced me to it.
Strong upvote because the divination rituals are really pseudo-RNG for when random choices are better than biased ones is an amazing insight, which I had not encountered previously.
I'm half way through the article and it's been an interesting read so far but I got to this sentence:
> But that is the trouble: we have no way to tell which traditions are adaptive and which are merely drift.
The article (so far) didn't provide evidence for that. I'd even say that the article provides some evidence against this claim. It describes a bunch of traditions, identifies them as useful, and explains why they're useful. I thik there are exaples of traditions that people identified as useless (or harmful). Like using torture to extract confessions (I hope this is an example old enough to not be controversial).
So far my impression is that the article makes a good case for "distinguishing useful traditions is hard" and provides a few examples of traditions for which reasons why they're good require way more knowledge than people executing those traditions have. Still saying it's impossible seems wrong.
On the other hand pointing out that we might invent a wrong explanation for a tradition (removing bitternes from manioc) and screw up the clean up process is a good point.
To me, this is exactly what the LW community (and the broader progressive tribe surrounding it) needs to hear. This post, along with other developments of thought in the same direction, has caused a major shift in how I think about changing things.
The first quote is most important, and I find myself using it quite often if I'm met with a person my age (or even older) that dismisses a tradition as obviously dumb. Why do you think the tradition exists in the first place? If you don't know, how can you be so sure it doesn't serve some function?
If a piece of code you ran raised an error, it would be pretty dumb to just remove the statement that raises the error to fix your program. The only reason why Omega doesn't break down that easily is because naive people like us have kicked at it long enough to make it a little more robust. That doesn't mean kicking it makes it stronger.
This direction of thinking caused a major shift in my career, from trying to hack away at the margins of society to becoming a part of the establishment. I'm writing this from my office at a governmental bank. This post is the reason that it's not some indie startup instead, and I don't regret it.
I also thought the insight of "divination rituals are really pseudo-RNG " very interesting. I think it would make an interesting research agenda for a number of grad students in various fields to explore. I would expand it further into the whole "role of faith" in society type of idea. Perhaps there is more to religions that we realize -- regardless of one's personal views on the existence of any god or gods.
Your link reminds me of an old econ article I read in school years ago (The Origins of Predictable Behavior, Ron Heiner) but seems to tease out an even more nuanced view of the challenges of knowledge and rules.
I found this valuable, especially in the context of thinking about how smart humans actually are compared with other animals, and how this should impact our expectations for AI. I generally agree that we are deeply reliant on culture, in ways that aren't obvious from the inside. I suspect that this is evidence in favour of slower takeoff speeds, because being as smart as humans isn't nearly enough to do as well as humans. Any other thoughts in this direction?
Interestingly, I read this as evidence in favour of a fast takeoff, because being able to explicitly sort through the traditions for which ones were valuable and which ones were not, seems something a decent AI could do much better than a human. If part of our knowledge and power is held in forms we don't understand, we become vulnerable to entities who do understand it.
Being able to speak is probably more important than being as smart as a human. Cultural / memetic evolution is orders of magnitude faster than biological, but its ability to function is dependent on having a memory better than mortal minds. Speech gives some limited non-mortal memory, as does writing the printing press, or the internet. These inventions enable more efficient evolution. AI will ramp up evolution to even higher speeds, since external memory will be replaced with internal 1) lossless and 2) intelligent memory. As such I am unconvinced that this would mean slower takeoff speeds. (You just explained that the most important factor in doing well as humans is something humans are not overly good at, instead of the special magic that only humans posess.)
I suspect that this is evidence in favour of slower takeoff speeds, because being as smart as humans isn't nearly enough to do as well as humans.
I don't see the connection between the latter claim and the former claim.