Human instincts, symbol grounding, and the blank-slate neocortex

[-]Charlie Steiner5yΩ360

This was just on my front page for me, for some reason. So, it occurs to me that the example of the evolved FPGA is precisely the nightmare scenario for the CCA hypothesis.

If neurons behave according to simple rules during growth and development, and there are only smooth modulations of chemical signals during development, then nevertheless you might get regions of the cortex that look very similar, but whose cells are exploiting the hardly-noticeable FPGA-style quirks of physics in different ways. You'd have to detect the difference by luckily choosing the right sort of computational property to measure.

[-]Steven Byrnes5y*Ω250

Thanks for the comment! When I think about it now (8 months later), I have three reasons for continuing to think CCA is broadly right:

Cytoarchitectural (quasi-) uniformity. I agree that this doesn't definitively prove anything by itself, but it's highly suggestive. If different parts of the cortex were doing systematically very different computations, well maybe they would start out looking similar when the differentiation first started to arise millions of years ago, but over evolutionary time you would expect them to gradually diverge into superficially-obviously-different endpoints that are more appropriate to their different functions.
Narrowness of the target, sorta. Let's say there's a module that takes specific categories of inputs (feedforward, feedback, reward, prediction-error flags) and has certain types of outputs, and it systematically learns to predict the feedforward input and control the outputs according to generative models following this kind of selection criterion (or something like that). This is a very specific and very useful thing. Whatever the reward signal is, this module will construct a theory about what causes that reward signal and make plans to increase it. And this kind of module automatically tiles—you can connect multiple modules and they'll be able to work together to build more complex composite generative models integrating more inputs to make better reward predictions and better plans. I feel like you can't just shove some other computation into this system and have it work—it's either part of this coordinated prediction-and-action mechanism, or not (in which case the coordination prediction-and-action mechanism will learn to predict it and/or control it, just like it does for the motor plant etc.). Anyway, it's possible that some part of the neocortex is doing a different sort of computation, and not part of the prediction-and-action mechanism. But if so, I would just shrug and say "maybe it's technically part of the neocortex, but when I say "neocortex", I'm using the term loosely and excluding that particular part." After all, I am not an anatomical purist; I am already including part of the thalamus when I say "neocortex" for example (I have a footnote in the article apologizing for that). Sorry if this description is a bit incoherent, I need to think about how to articulate this better.
Although it's probably just the Dunning-Kruger talking, I do think I at least vaguely understand what the algorithm is doing and how it works, and I feel like I can concretely see how it explains everything about human intelligence including causality, counterfactuals, hierarchical planning, task-switching, deliberation, analogies, concepts, etc. etc.

[-]Дмитрий Зеленский5y60

For me, your examples of why visual perception needs the same things as language, including time window, is a standard, textbook-level (and often used!) proof of the fact they're both (widely understood) Fodorian modules (in case of visual processing, two distinct modules indeed, though the labels "conscious" and "subconscious" are strange, I'm used to calling those "What-path" and "Where-path"), fine-tuned but not fully designed during the time-window, not that they are, vice versa, both handled by general algorithm like a snowflake.

Now, I understand that Fodorian modules (even when you throw away the old requirement of there being a strictly limited part of the cortex responsible for it) are not that widely held nowadays. However, when I look at people, I cannot help seeing them. From prosopagnosia to specific language impairments, aka aphasias (only two of the six commonly discussed aphasias are really language-based but the name stuck) to memory disruptions, we see individual modules breaking - including in-born reaking, before fine-tuning! - and just as well we see people whose general intelligence is reasonably low with unusually good performance of some of their modules.

Addendum: "visual" in "visual processing" is, of course, a red herring. It would be better to speak of two perception modules, with input variable (blindborn people fine-tune it to other things, for example - whereas those blinded in adulthood, AFAIK, do not).

[-]Steven Byrnes5y40

your examples of why visual perception needs the same things as language, including time window, is a standard, textbook-level (and often used!) proof of the fact they're both (widely understood) Fodorian modules

Interesting! Can you point me to a textbook or other reference that makes this argument?

the labels "conscious" and "subconscious" are strange, I'm used to calling those "What-path" and "Where-path"

If you're talking about the dorsal and lateral streams, that's not what I'm talking about. The dorsal steam and lateral stream are both part of the neocortex. I was talking about "vision processing in the neocortex" versus "vision processing in the midbrain (especially superior colliculus)". Does that make sense?

[-]Дмитрий Зеленский5y30

On the second point - I have misunderstood you, now I see what you're talking about. If Fodorian modules' view is right, the neocortex one(s) still isn't (aren't) "conscious". The received wisdom I have says that modules are:

1)Automatic (one cannot consciously change how they work - except by cutting off their input) - hence susceptible to illusions/wrong analyses/...;

2)Autonomous (consciousness only "sees" outputs, a module is black box for its owner; these two properties are related but distinct - yet something that has both can barely be called "conscious");

3)Inherited with a critical period of fine-tuning (that's basically what you called time window).

There were some more points but I (obviously) forgot them. And that brings me to your first point: I can't point to a textbook right away but that was part of several courses I was taught (Psychology of cognitive processes at Moscow State University (Fundamental and Applied Linguistics program); Language, Music, and Cognition in NYI 2016 - nyi.spb.ru).

[-]Steven Byrnes5y*40

Thank you! I agree that much of what happens in the neocortex is not conscious, and agree with your 1+2+3. I have edited the wording of that sentence to be less confusing. As for what is and isn't conscious, I like the Global Neuronal Workspace idea. I use that a lot when thinking about the neocortex.

I think I used the term "sensitive windows" in footnote 2 for what you call "critical period of fine-tuning". I was thinking of things like low-level sensory processing, the fact that you can't learn a language in the native accent as an adult, etc. Then I also talked about "time windows" in the context of filial imprinting.

I used two different terms, "sensitive windows" and "time windows", but we may ask: are these two different things, or two examples of the same thing? I'm not sure. I would guess that they're the same mechanism, but the former is locking in "normal" information-processing connections (from neocortex to neocortex), and the latter is locking in specifically connections to dopamine neurons or connections to other hormones or some other type of connection to the limbic system. I'm still trying to understand how those latter connections work...

[-]Дмитрий Зеленский5y40

I would think that the former are the _mechanism_ of the latter - though, as they say, "don't quote me on that".

There is an interesting question of whether, if many things are modules, there is also non-module part, the "general intelligence" part which does not share those properties. Perhaps unsurprisingly, there is no consensus (though my intuitions say there is the GI part).

Also, it seems that different modules might use the same (common) working memory - though this is not set in stone (and depends, in particular, on your analysis of language - if late Chomsky is right, only phonology (PF) and perhaps semantics (LF) are modular, whereas syntax uses our general recursive ability, and this is why it uses general working memory).

[-]Steven Byrnes5y80

Hmm, interesting!

My thinking about general intelligence is the combination of

Common Cortical Algorithm—every part of the neocortex is a similarly-constructed general-purpose generative model builder that basically does some combination of self-supervised learning, RL, and Model Predictive Control
Specialized Functionality Via The Neocortex's Gross Wiring Diagram—So if you seed a region with particular feedforward and feedback information streams, then it will find patterns and build up the corresponding concept space.
...and therefore, that there are lots of different neocortical modules exactly to the extent that there are lots of different neocortical regions, with strict borders between them and few axons crossing those borders. I'm not sure the extent to which this is the case.
Global Neuronal Workspace (GNW) connecting diverse parts of the neocortex to each other and to long-term memory.
"System 2" is just a way of stringing together various automatic thoughts (generative models) into a sorta ad-hoc serial computer.

Also, I'm not convinced that "general working memory" is a thing. My understanding is that we can do things like remember a word by putting it on loop using speech motor control circuits. But I would expect that someone with a lesion to those speech motor control circuits would still be able to, say, hold an image in their head for a short period of time using visual cortex. I think recurrent networks are ubiquitous in the neocortex, and any of those networks can potentially hold an activation at least for a couple seconds, and a few of them for much longer. Or maybe when you're thinking about "general working memory", you're really thinking of the GNW?

[-]Дмитрий Зеленский5y40

1. "My understanding is that we can do things like remember a word by putting it on loop using speech motor control circuits" - this is called phonological loop in psycholinguistics (psychology) and is NOT THE SAME as working memory - in fact, tests for working memory usually include reading something aloud precisely to occupy the circuits and not let the test subject take advantage of their phonological loop. What I mean by working memory is the number of things one can hold in their mind simultaneously captured by "5+-2" work and Daneman's tests - whatever the explanation is.

2. Fodorian modules are, by definition, barely compatible with CCA. And the Zeitgeist of theoretical linguistics leads me to think that when you use RNN to explain something you're cheating your way to performance instead of explaining what goes on (i.e. to think that brain ISN'T an RNN or a combination thereof - at least not in an obvious sense). Thus we don't quite share neurological assumptions - though bridging to a common point may well be possible.

[-]Steven Byrnes5y30

this is called phonological loop

Thank you for clearing up my confusion! :-)

when you use RNN to explain something...

To be clear, I am using the term "recurrent" as a kinda generic term meaning "having a connectivity graph in which there are cycles". That's what I think is ubiquitous in the neocortex. I absolutely do not think that "the kind of RNN that ML practitioners frequently use today" is similar to how the neocortex works. Indeed, I think very few ML practitioners are using algorithms that are fundamentally similar to brain algorithms. (I think Dileep George is one of the exceptions.)

Fodorian modules are, by definition, barely compatible with CCA

...unless the Fodorian modules are each running the same algorithm on different input data, right?

[-]Дмитрий Зеленский5y30

Oh, then sorry about the RNN attack ;)

Well, no. In particular, if you feed the same sound input to linguistic module (PF) and to the module of (say, initially visual) perception, the very intuition behind Fodorian modules is that they will *not* do the same - PF will try to find linguistic expressions similar to the input whereas the perception module will try to, well, tell where the sound comes from, how loud it is and things like that.

[-]Gordon Seidoh Worley6y50

I found this post really interesting. My main interest has been in understanding human values, and I've been excited by predictive coding because it possibly offers a way to ground values (good being derived from error minimization, bad from error maximization). The CCA theory could help explain why it seems so much of the brain is "doing the same kind of thing" that could result in predictive coding being a useful model even if it turns out the brain doesn't literally have neurons wired up as control systems that minimize sensory prediction error.

[-]Abe Dillon5y60

Hey, G Gordon Worley III!

I just finished reading this post because Steve2152 was one of the two people (you being the other) to comment on my (accidentally published) post on formalizing and justifying the concept of emotions.

It's interesting to hear that you're looking for a foundational grounding of human values because I'm planning a post on that subject as well. I think you're close with the concept of error minimization. My theory reaches back to the origins of life and what sets living systems apart from non-living systems. Living systems are locally anti-entropic which means: 1) According to the second law of thermodynamics, a living system can never be a truly closed system. 2) Life is characterized by a medium that can gather information such as genetic material.

The second law of thermodynamics means that all things decay, so it's not enough to simply gather information, the system must also preserve the information it gathers. This creates an interesting dynamic because gathering information inherently means encountering entropy (the unknown) which is inherently dangerous (what does this red button do?). It's somewhat at odds with the goal of preserving information. You can even see this fundamental dichotomy manifest in the collective intelligence of the human race playing tug-of-war between conservatism (which is fundamentally about stability and preservation of norms) and liberalism (which is fundamentally about seeking progress or new ways to better society).

Another interesting consequence of the 'telos' of life being to gather and preserve information is: it inherently provides a means of assigning value to information. That is: information is more valuable the more it pertains to the goal of gathering and preserving information. If an asteroid were about to hit earth and you were chosen to live on a space colony until Earth's atmosphere allowed humans to return and start society anew, you would probably favor taking a 16 GB thumb drive with the entire English Wikipedia article text than a server-rack full several petabytes of high-definition recordings of all the reality television ever filmed, because that won't be super helpful toward the goal of preserving knowledge *relevant* to man kind's survival.

The theory also opens interesting discussions like, if all living things have a common goal; why do things like paracites, conflict, and war exist? Also, how has evolution led to a set of instincts that imperfectly approximate this goal? How do we implement this goal in an intelligent system? How do we guarantee such an implementation will not result in conflict? Etc.

Anyway, I hope you'll read it when I publish it and let me know what you think!

[-]Gordon Seidoh Worley5y20

Looking forward to reading it. In the meantime, if you didn't stumble on them already, you might enjoy these posts I wrote as I think they point to some similar things:

[-]Charlie Steiner6y40

This is a really cool post, thanks!

[-]Ness Blackbird4mo30

This list has been the focus of heavy criticism since its publication in 1991. According to Wikipedia (https://en.wikipedia.org/wiki/Cultural_universal), the scholarship was not up to scratch.

[-]hold_my_fish4y30

I find myself returning to this because the idea of a "common cortical algorithm" is intriguing.

It seems to me that if there is a "common cortical algorithm" then there is also a "common cortical problem" that it solves. I suspect it would be useful to understand what this problem is.

(As an example of why isolating the algorithm and problem could be quite different, consider linear programming. To solve a linear programming problem, you can choose a simplex algorithm or an interior-point method, and these are fundamentally different approaches that are both viable. It's also quite a bit easier to state linear programming as a problem than it is to describe either solution approach.)

Do you have a view on the most plausible candidates for a "common cortical problem" (CCP)? The tricky aspects that come to mind: not being too narrow (i.e. the CCP should include (almost) everything the CCA can do), not being too broad (i.e. the CCA should be able solve (almost) every instance of the CCP), and not being too vague (ideally precise enough that you could actually make a benchmark test suite to evaluate proposed solutions).

[-]Steven Byrnes4y30

Thanks!

The way I'm currently thinking about it is: In everywhere but the frontal lobe, the task is something like

Find a generative model that accurately predicts Input Signal X based on Contextual Information Y.

But it's different X & Y for different parts of the cortex, and there can even be cascades where one region needs to predict the residual prediction error from another region (ref). And there's also a top-down attention mechanism such that not all prediction errors are equally bad.

The frontal lobe is a bit different in that it's choosing what action to take or what thought to think (at least in part). That's not purely a prediction task, because it has more than one right answer. I mean, you can predict that you'll go left, then go left, and that's a correct prediction. Or you can predict that you'll go right, then go right, and that's a correct prediction too! So it's not just predictions; we need reinforcement learning / rewards too. In those cases, the task is "Find a generative model that is making correct predictions AND leading to high rewards," presumably. But I don't think that's really something that the neocortex is doing, per se. I think it's the basal ganglia (BG), which sends outputs to the frontal lobe. I think the BG looks at what the neocortex is doing, calculates a value function (using TD learning, and storing its information in the striatum), and then (loosely speaking) the BG reaches up into the neocortex and fiddles with it, trying to suppress the patterns of activity that it thinks would lead to lower rewards and trying to amplify the patterns of activity that it thinks would lead to higher rewards.

See my Predictive coding = RL + SL + Bayes + MPC for my old first-cut attempt to think through this stuff. Meanwhile I've been reading all about the striatum and RL stuff, more posts forthcoming I hope.

Happy for any thoughts on that. :-)

[-]hold_my_fish4y10

Thanks for your reply!

A few points where clarification would help, if you don't mind (feel free to skip some):

What are the capabilities of the "generative model"? In general, the term seems to be used in various ways. e.g.
- Sampling from the learned distribution (analogous to GPT-3 at temp=1)
- Evaluating the probability of a given point
- Producing the predicted most likely point (analogous to GPT-3 at temp=0)
Is what we're predicting the input at the next time step? (Sometimes "predict" can be used to mean filling in missing information, but that doesn't seem to make sense in this context.) Also, I'm not sure what I mean by "time step" here.
The "input signal" here is coming from whatever is wired into the cortex, right? Does it work to think of this as a vector in ?
Is the contextual information just whatever is the current input, plus whatever signals are still bouncing around?

Also, the capability described may be a bit too broad, since there are some predictions that the cortex seems to be bad at. Consider predicting the sum of two 8-digit integers. Digital computers compute that easily, so it's fundamentally an easy problem, but for humans to do it requires effort. Yet for some other predictions, the cortex easily outperforms today's digital computers. What characterizes the prediction problems that the cortex does well?

[-]Steven Byrnes4y30

Think of a generative model as something like "This thing I'm looking at is a red bouncy ball". Just looking at it you can guess pretty well how much it would weigh if you lifted it, how it would feel if you rubbed it, how it would smell if you smelled it, and how it would bounce if you threw it. Lots of ways to query these models! Powerful stuff!

some predictions that the cortex seems to be bad at

If a model is trained to minimize a loss function L, that doesn't mean that, after training, it winds up with a very low value of L in every possible case. Right? I'm confused about why you're confused. :-P

[-]Vermillion6y10

Thank you very much for this, I had heard of CCA theory but didn't know enough to evaluate it myself. I think this opens new possible paths to AGI I had not thoroughly considered before.

[-]Steven Byrnes6y20

Thanks for saying that! :)

[-]countedblessings6y10

Any thoughts on the relevance of category theory as the same kind of "universal modeling system" for mathematics as the brain might well be for real life?

[-]Steven Byrnes6y30

Don't know much about category theory, guess I'll have to read your posts :)

I'll be talking a lot about the neocortex in this article, but shout-out to the thalamus and hippocampus, the other two primary parts of the brain's predictive-world-model-building-system. I'm just leaving them out for simplicity; this doesn't have any important implications for this article. ↩︎
More examples of region-to-region variation in the neocortex that are (plausibly) genetically-coded: (1) Spindle neurons only exist in a couple specific parts of the neocortex. I don't really know what's the deal with those. Kurzweil claims they're important for social emotions and empathy, if I recall correctly. Hmmm. (2) "Sensitive windows" (see Dehaene): Low-level sensory processing areas more-or-less lock themselves down to prevent further learning very early in life, and certain language-processing areas lock themselves down somewhat later, and high-level conceptual areas don't ever lock themselves down at all (at least, not as completely). I bet that's genetically hardwired. I guess psychedelics can undermine this lock-down mechanism? ↩︎
I have heard that the primary motor cortex is not the only part of the neocortex that emits motor commands, but don't know the details. ↩︎
Also, people who lose various parts of the neocortex are often capable of full recovery, if it happens early enough in infancy, which suggests to me that the CCA's wiring-via-learning capability is doing most of the work, and maybe the innate wiring diagram is mostly just getting things set up more quickly and reliably. ↩︎
See Rethinking Innateness p116, or better yet Johnson's article ↩︎
See, for example, Fast Detector/First Responder: Interactions between the Superior Colliculus-Pulvinar Pathway and Stimuli Relevant to Primates. Also, let us pause and reflect on the fact that humans have two different visual processing systems! Pretty cool! The most famous consequence is blindsight, a condition where the ~~subconscious~~ midbrain vision processing system (superior colliculus) is intact but the ~~conscious~~ neocortical visual processing system is not working. This study proves that blindsighted people can recognize not just faces but specific facial expressions. I strongly suspect blindsighted people would react to snakes and spiders too, but can't find any good studies (that study in the previous sentence regrettably used stationary pictures of spiders and snakes, not videos of them scampering and slithering). ↩︎

LESSWRONG
LW

LESSWRONG
LW

62

Human instincts, symbol grounding, and the blank-slate neocortex

62

Ω 23

62

Ω 23

Intro: What is Common Cortical Algorithm (CCA) theory, and why does it matter for AGI?

Should we believe CCA theory?

CCA theory vs human-universal traits and instincts

List of ways that human-universal instincts and behaviors can exist despite CCA theory

Conclusion