•

Created by Zack_M_Davis at 3y

Zack_M_Davis v1.5.0Nov 10th 2009 (+11/-8) LaTeX formatting giving error message instead; back to plaintext until wiki server can be updated 2

For example, given an object that can either have or not have each of ~~N~~*n* properties, there are ~~2~~^{2^Nn} different descriptions corresponding to the possible objects of that kind (a number exponential in ~~N)~~*n*). The number of possible concepts, each of which either includes a given description or doesn't, is one exponential higher: ~~2~~^{2^(}~~2~~^{2^Nn})

In order to do inference, we constantly need to make use of categories and concepts: it is neither possible nor desirable to deal with every unique arrangement of quarks and leptons on an individual basis. Fortunately,~~ because we don't live in a maximum-entropy universe of absolute chaos,~~ we can talk about repeatable higher-level regularities in the world instead: we can distinguish particular configurations of matter as instantiations of object concepts like *chair* or *human*, and say that these objects have particular properties, like *red* or *alive*.

The sheer number of distinct configurations in which matter could be arranged is unimaginably vast, but the **superexponential conceptspace** of the number of different ways to *categorize* these possible objects is even vaster.~~ If (for purposes~~

For example, given an object that can either have or not have each of ~~exposition)~~N properties, there are 2^{nN} different descriptions corresponding to the possible objects of that kind (a number exponential in ~~the world which either ~~~~are~~~~, or are ~~~~not~~~~ instantiations of some concept, then the~~N). The number of possible ~~concepts~~concepts, each of which either includes a given description or doesn't, is ~~2^~~one exponential higher: 2^{(}2^{nN (for})

Without an inductive bias, restricting attention to only a small portion of possible concepts, it's not possible to navigate the ~~mathematics involved,~~conceptspace: to learn a concept, a "fully general" learner would need to see all the individual examples that define it. Using ~~powerset~~probability~~). Most~~ to mark the extent to which each possibility belongs to a concept is another approach to express prior information and its control over the process of ~~these possible concepts are complicated enough to be ruled out ~~~~a priori~~~~ by your prior; you can't expect to encounter enough evidence to cut down such a large space. The work of proper inference is to "carve reality at its joints"; to find simple generalizations and simple concepts that let you make useful inferences with respect to your goals.~~learning.

In order to do inference, we constantly need to make use of categories and concepts: it is neither possible nor desirable to deal with every unique arrangement of quarks and leptons on an individual basis. Fortunately, because we don't live in a maximum-entropy universe of absolute chaos, we can talk about repeatable higher-level regularities in the ~~world:~~world instead: we can distinguish particular configurations of matter as instantiations of object concepts like *chair* or *human*, and say that these objects have particular properties, like *red* or *alive*.

~~The sheer number of distinct configurations in which matter could be arranged is ~~~~unimaginably~~~~ vast, but the ~~~~superexponential conceptspace~~~~ of the number of different ways to ~~~~categorize~~~~ these possible objects is even vaster.~~

In order to do inference, we constantly need to make use of categories and concepts: it is neither possible nor desirable to deal with every unique arrangement of quarks and leptons on an individual basis. Fortunately, because we don't live in a maximum-entropy universe of absolute chaos, we can~~ do this, and~~ talk about repeatable higher-level regularities in the world: we can distinguish particular configurations of matter as instantiations of object concepts like *chair* or *human*, and say that these objects have particular properties, like *red* or *alive*.

The sheer number of distinct configurations in which matter could be arranged is unimaginably vast, but the **superexponential conceptspace** of the number of different ways to *categorize* these possible objects is even vaster. If (for purposes of exposition) there are *n* objects in the world which either *are*, or are *not* instantiations of some concept, then the number of possible concepts is 2^*n* (for the mathematics involved, see powerset). Most of these possible concepts are complicated enough to be ruled out *a priori* by your prior; you can't expect to encounter enough evidence to cut down such a large space. The work of proper inference is to "carve reality at its joints"; to find simple generalizations and simple concepts that let you make useful inferences with respect to your goals.

The sheer number of distinct configurations in which matter could be arranged is unimaginably vast, but the **superexponential conceptspace** of the number of different ways to *categorize* these possible objects is even vaster.

In order to do inference, we constantly need to make use of categories and concepts: it is neither possible nor desirable to deal with every unique arrangement of quarks and leptons on an individual basis. Fortunately, we don't live in a maximum-entropy universe of absolute chaos, we can do this, and talk about repeatable higher-level regularities in the world: we can distinguish particular configurations of matter as instantiations of object concepts like *chair* or *human*, and say that these objects have particular properties, like *red* or *alive*.

The sheer number of distinct configurations in which matter could be arranged is unimaginably vast, but the

superexponential conceptspaceof the number of different ways tocategorizethese possible objects is even vaster.## What are concepts?

~~The sheer number of~~~~distinct configurations~~~~in which matter could be arranged is~~~~unimaginably~~~~vast, but the~~~~superexponential conceptspace~~~~of the number of different ways to~~~~categorize~~~~these possible objects is even vaster.~~~~Blog posts~~Key Posts