Episode 28: Convergence to Relevance Realization
So last time I went through with you a series of arguments trying to show you the centrality of the issue of relevance realization. I want to review that with you and then try to begin with an account of how we might come up with a naturalistic explanation of relevance realization and then build that into an overall plausibility argument about using that notion of relevance realization to explain many of the features that we consider central to human spirituality, meaning-making, self-transcendence, altered states of consciousness, and wisdom.
Before I begin that I want to remind everybody of how much the work I'm talking about now has been done in collaboration with other people, especially the work with Tim Lillicrap and Blake Richards in 2012, the article we published in the journal of logic and computation with Leo Ferraro in 2013, some current work with Leo Ferraro, Anderson Todd, and Richard Wu, current work I'm doing with Christopher Mastropietro, and some past work with Zachary Irving and Leo Ferraro on the nature of intelligence.
We did a series of argument that pointed towards how central relevance realization is. We did arguments around the nature of problem-solving, and remember we saw the idea there of search space as proposed by Newell and Simon and we faced a couple of important issues there. We faced issues of combinatorial explosion, and what we need is problem formulation or problem framing that allows us to avoid combinatorial explosion by zeroing in on relevant information. I also propose to you (and I'll return to this later) that problem-solving is our best way of trying to understand what we mean by intelligence: your capacity as being a general problem-solver.
Also we noted the problem of ill-definedness, very often a problem formulation is needed in order to determine what the relevant information is and what the relevant structure of the information is so that again points us into relevance. These two together also pointed towards a phenomena we have already talked about (insight) and the fact that you often have to solve a problem by altering your problem formulation and re-determining what you consider relevant.
We then took a look at categorization (I'll come back to this again in another way a little bit later in this lecture), how it ultimately depends on judgments of similarity and we can get into an equivocation there. We can equivocate between a purely logical notion of similarity in which case any two objects are indefinitely similar or dissimilar to each other and if we mean instead of logical similarity (which would not help us to categorize) psychological similarity then we're talking about making a comparison of two things in terms of the relevant features of comparison, the relevant aspects. So we're into relevance and we're also introducing an important idea I want you to remember, this notion of an 'aspect', a set of relevant features that cohere together and are relevant to us, especially in projects like categorization.
Doing good cog-sci, I do a convergence argument to get a trustworthy problem or construct and then I basically do a divergence argument to show how it has the potential to explain many important phenomena and establish a relevant balance between them, and so that's what I'm building you. Right now we're on this side, how all things are converging on relevance realization and then we can use this to explain many of the features that seem to be central to human spirituality, meaning-making, self-transcendence, altered states of consciousness, and wisdom.
We took a look at communication, and we saw the issue here is that the fact you have to convey more than you can say and then that led us into the work of Grice and a series of maxims that make conversational implicature possible. All of the maxims collapsed to the maxim of being relevant.
We took a look at robotics, the actual interaction with the environment. Here's the idea of being an agent. We saw the robot was trying to pull the battery that's also on the wagon and that wagon also has the bomb on it and what we saw is the problem of the proliferation of side effects. You can't ignore all the side effects or you'll be grotesquely stupid, you can't check all side effects or you'll be grotesquely incapable, and so therefore you have to zoom in on their relevant side effects. So again and again and again everything is centering on this.
As an autistic person at parts he seemed to argue that neurotypicality would be at the core of being intelligent.
The social situations (like asking for gas) he described he presented them as not working to never happen. However the ire and tensions when it doesn't go as planned seemed very familiar to me. As the person giving advice I would have probably leaned more on using explicit distance descriptors (such as 250 meters) rather than nearby corner.
Connecting to how he was previously saying things he might actually mean that in order to be rational to ne... (read more)
John Vervaeke has a lecture series on YouTube called Awakening from the Meaning Crisis. I thought it was great, so I'm arranging a lecture club to discuss it here on Less Wrong. The format is simple: each weekday I post a comment that's a link to the next lecture and the summary (which I plan on stealing from the recap at the beginning of the next lecture), and then sometimes comment beneath it with my own thoughts. If you're coming late (even years late!) feel free to join in, and go at whatever pace works for you.
(Who is John Vervaeke? He's a lecturer in cognitive science at the University of Toronto. I hadn't heard of him before the series, which came highly recommended to me.)
I split the lecture series into three parts: the philosophical, religious, and cultural history of humankind (25 episodes) related to meaning, the cognitive science of wisdom and meaning (20 episodes), and more recent philosophy related to the meaning crisis specifically (5 episodes). Each episode is about an hour at regular speed (but I think they're understandable at 2x speed). I am not yet aware of a good text version of the lectures; I also have some suspicion that some important content is not in the text itself, and so even if I transcribed them (or paid someone to) it'd still be worth watching or listening to it.
I think the subject matter is 1) very convergent with the sort of rationality people are interested in on LW, and 2) relevant to AI alignment, especially thinking about embedded agency.
Discussion: