Concept Safety
Multiagent Models of Mind
Keith Stanovich: What Intelligence Tests Miss


All is fair in love and war, on Zero-sum games in life

More specifically, interpersonal interaction has both a dominance dimension ("of status, dominance, power, ambitiousness, assertiveness, or control") and a warmth dimension ("of agreeableness, compassion, nurturant, solidarity, friendliness, warmth, affiliation or love"). Dominance is zero-sum, but warmth is not.

Cultures also vary in how much they emphasize the dominance and warmth dimensions. In more "status-flat" cultures (such as the Nordic countries), social conventions tend to de-emphasize status differences, making relative status less important and letting the warmth dimension matter more.

It seems interesting to me that I feel like I mostly encounter arguments such as "status is zero-sum so we can't ever make everyone happy" expressed by people from non-Nordic countries. The notion always seemed unintuitive to me, and I don't think that the reason is just "Kaj personally pays less attention to dominance status" since I do feel pretty sensitive to it. Rather, it feels like a significant part of it is Finnish culture just not caring about dominance status that much, relative to warmth, making it hard for me to see why the zero-sumness of status should necessarily be a significant problem.

Why I Work on Ads

From the perspective of users, I think the internet would be essentially unusable unless you subscribed to a few standard services, which would then have harmful levels of leverage. 

I wonder about that: before third-party services started popping up, internet service providers and nonprofits used to offer more services that are now offered by third parties. E.g. your ISP used to give you an e-mail account and website space, and services such as Usenet and IRC functioned in a decentralized fashion, with servers being hosted by universities, ISPs and others. That model won't work for everything, but it doesn't seem like too much of a stretch to imagine services such as social media and search shifting to a more decentralized model if advertising was banned. (Decentralized social media networks such as Diaspora already exist; I'm under the impression that the main reason they're not used more is that network effects create too much lock-in to existing, more centralized services.)

Your Dog is Even Smarter Than You Think

From reading the discussion, I get the impression that some of the commenters are writing from a position of "the prior is against significant dog intelligence, and the evidence here could have alternative explanations, so I'm skeptical". That is, the people feel that it would be quite surprising if dogs really were this intelligent, so establishing it requires some pretty compelling evidence.

At the same time, my own feeling is more like "there's no strong prior against significant dog intelligence, and the evidence here could have alternative explanations, so I'm being tentatively open to this being real". As in, even if you hadn't shown me these videos, dogs being approximately as intelligent as described would have seemed like an entirely plausible possibility to me.

If there are any people who feel like my first paragraph does describe them, I'd be curious to hear why they're coming into this discussion with such a strong prior against dog intelligence.

If I had to articulate the reasons for my own "seems plausible" prior, they'd be something like:

  • a general vague sense of animal research tending to generally show that animals are smarter than people often think
  • some of the animals in books like "Don't Shoot the Dog" sounding relatively smart (e.g. vaguely recall the author mentioning that training often speeds up once the animal figures out that it's being trained, since then it can explicitly try to figure out what the trainer is trying to reward it for)

I once videotaped a beautiful Arabian mare who was being clicker-trained to prick her ears on command, so as to look alert in the show ring. She clearly knew that a click meant a handful of grain. She clearly knew her actions made her trainer click. And she knew it had something to do with her ears. But what? Holding her head erect, she rotated her ears individually: one forward, one back; then the reverse; then she flopped both ears to the sides like a rabbit, something I didn't know a horse could do on purpose. Finally, both ears went forward at once. Click! Aha! She had it straight from then on. It was charming, but it was also sad: We don't usually ask horses to think or to be inventive, and they seem to like to do it.

  • even relatively simple AI systems exhibiting surprisingly intelligent behavior (GPT-3), suggesting that there isn't necessarily a sharp distinction between human and less-than-human intelligence
Why I Work on Ads

The type of ad that merely informs you of an existence of a product is possible in theory, and maybe existed in 19th century, but I was born too late for that.

I have purchased clothes, plush animals, books, and games because of online advertisements that told me about their existence; I would have been unaware of the products in question if not for the ads. (I have also generally been happy with the products that I got; one of the clothes that I ordered is probably my favorite piece of clothing.)

Announcing The Inside View Podcast

CastingWords at least used to be accurate (the one time I used them, I don't recall the transcript having had any flaws).

Your Dog is Even Smarter Than You Think

So the danger is over-interpreting their output. Was this sentence intended or just random babbling? Does the dog understand the word differently from what it means in English? E.g. "bye" seems to become a verb meaning "leaving", "love you" is used for affection but obviously doesn't reflect deep understanding of the human concept of love. 

Interestingly, there's an argument that human infants also learn language by their parents over-interpreting their input, with the infants then adopting those interpretations as true. So one could argue that even if over-interpretation happens with dogs, that only makes it a process similar to human language learning, with the parent/child and owner/dog creating a shared language game.

Our first type of example comes from our own data concerning Zulu infants of between three and four months of age interacting with their mothers, and suggests an answer to this question. [...]

As noted above, there are times when a caregiver will want an infant to fall silent, or in isiZulu to 'thula'. Zulu children are traditionally expected to be less socially active than contemporary Western children, to initiate fewer interactions, and, crucially, to show a respectful attitude towards adults. An early manifestation of this is in behaviours where a mother attempts to make an infant keep quiet, sometimes saying 'thula' ('quiet'), 'njega' ('no'), while simultaneously gesturing, moving towards or away from the infant, and reacting to details of the infants own behaviour (see Cowley et al., in press).

At these times the mother regularly leans forward, so that more of the infants visual field is taken up by her face and palms. New vocalisations, and movements or re-orientations of gaze by the infant, are often 'nipped in the bud' by dominating vocalisations (sometimes showing prosodic properties indicative of disapproval, comforting, attention and/or arousal towards the mother herself) from the mother, sometimes accompanied by increasingly emphatic hand-waving, and even closer crowding of the infants visual field. [...]

With high regularity, and within relatively little time, the particular infant often does 'thula', at which point it is generally rewarded with smiling, gentle touching, and other comforting.

At this stage there is no reason to believe that the infant knows what 'thula' or 'njega' means, or even that it could reliably re-identify the words, let alone produce or contemplate them, so it is extremely unlikely that the word-based aspects of maternal utterance-activity provide labels for the infant. We are considering infants before the stage linguists call 'babbling', let alone recognisable speech production. It is not even necessary to suppose that it 'knows' that it is supposed to be quiet when behaved at in the ways we have just described. We know that the mother wants the child to be quiet, that this expresses itself in behaviour by the mother, and that the infant comes to be quiet.

If we examine the mothers behaviour, though, we can make sense of it. She ensures that it is difficult for the infant to attend to anything else by crowding its visual field. She rejects active or new behaviours on its part by cutting off its vocalisations and movements with dominating signals of her own. She largely restricts approval signals, including relaxing the crowding, and reducing the magnitude of her gesturing, as well as expressing comfort through vocalisation, facial signalling and touch, to moments when the infant begins to quieten down. Its not particularly surprising, then, that it does quieten down.

The mothers behaviour includes salient, repeated, features which are apt for learning. Her patterns of hand gesturing, for example, could at the outset be iconic of the whole episode including her behaviour and the infants becoming quiet, but, when repetition allows the gesture to be individuated and recognised in its own right, go on to become an indexical cue that quietness should follow. The infants responses then become indexical for the mother of the degree to which the child is co-operative, well-behaved, or, more plainly, 'good'. Caregiver descriptions of infant behaviour at these times, manifest either in their explicit vocalisations to the child, including references to being 'good', or references to possible disciplinary sanctions such as 'kuza baba manje' ('wheres your father now?') or, in interviews following the videotaping, show that infant behaviour even at this early age is being classified in line with culturally specific expectations of good and bad behaviour. And a crucial part of what makes for a 'good' child is responding in ways sensitive to what caregiver behaviour is actually about, strikingly in controlling episodes such as the one just described, which make possible the earliest ascriptions of 'obedience', 'cooperativeness' and so forth.

These ascriptions are over-interpretations. They are, though, necessary overinterpretations, in so far as they motivate caregivers to imbue their own behaviour with regularities manifest regularities in their own behaviour which are then available as structure in the interactional environment for (learning by) the infant. A further episode from our data, in this case concerning a child of around four months, illustrates this point about over-interpretation. In it an infant repeatedly vocalises in ways which to its mother, at least, are suggestive of its saying 'up'. Each time she says 'up'?, or 'you want to go up'? and after a few repetitions she lifts the child. Prior to the lifting, there is little evidence that the child actually wants to be lifted, or that it has its attention focussed on anything in particular, except perhaps its own experiments in vocal control. When it is lifted, though, it beams widely. Whatever it did want, if anything, it is now, we suggest, one step closer to figuring out how to behave in ways that lead to its being lifted up.

Still on the subject of lifting, consider the common gesture made around the eighth month by infants who want to be picked up (that is, who subsequently smile or otherwise show approval when they are picked up following such a gesture): a simultaneous raising, or flapping, of both arms (see Lock, 1991). This gesture is not simply copied from common adult behaviours. In the terms we are using here it is partly iconic, in virtue of being a common posture of infants while they are in fact being held up, and partly indexical, in virtue of being able to stand on its own as an indicator of 'being up', as well as being symbolically interpretable as an invitation to lift, or a request to be lifted. Such gestures are, importantly, serviceable label candidates, in virtue of being amenable to disembedding from behaviour, and eventually coming under deliberate control. An infant need not want to be lifted the first few times it makes such a gesture, it has only to be able to notice that the gesture tends to be followed by liftings.

If and when such learning takes place, it does so in the affectively charged environment we have briefly described. We want to bring discussion of the current example to a close by suggesting a way in which these interactions should be regarded as a further example of how minds can be extended through action. Clark and Chalmers suggestion is that paradigmatically mental states and processes can be realised by structures and resources external to the brain. The world beyond the skull of any individual includes, of course, the skulls and brains of others. If active externalism motivates the recognition of a cognitive prosthesis such as a filofax as part of what realises a mind, then the embodied brain of another can also play that role. Here, then, is our suggestion: that at times interacting caregiver-infant dyads are neither one individual nor two, but somewhere in between. At the risk of sounding sensational and un-PC at the same time, infant brains can be temporarily colonised by caregivers so as to accelerate learning processes. [...]

The instances of indexical learning we describe also permit the beginning of a kind of 'semiotic arms race' between infants and caregivers. Once an infant has learned, for example, that the arms-up gesture can lead to being lifted, it is possible for 'requests' (that is, behaviours taken as requests by others, no matter how they are to the infant) to be lifted to be acted on, or to be refused. Prior to the construction and learning of the indexical relationship, this was impossible––a parent would lift a child when the parent wanted to, or thought it would serve some end. Once it has been learned, 'requests' can be differentially responded to, depending on their situation in patterns of interaction extending through time. Personal and cultural contingencies about infants and parents will codetermine what patterns are formed, and whether, for example, requested lifting is more likely after relatively quick acquiescence to silencing behaviour, or less likely in the period following failure to attend to objects or events in which a caregiver attempted to arouse interest.

Death by Red Tape

In the Zones of Thought universe, there is a cycle of civilization: civilizations rise from stone-age technology, gradually accumulating more technology, until they reach the peak of technological possibility. At that point, the only way they can improve society is by over-optimizing it for typical cases, removing slack. Once society has removed its slack, it's just a matter of time until unforeseen events force the system slightly outside of its safe parameters. This sets off a chain reaction: like dominoes falling, the failure of one subsystem causes the failure of another and another. This catastrophe either kills everyone on the planet, or sets things so far back that society has to start from scratch.

Reminds me of this:

In 1988, Joseph Tainter wrote a chilling book called The Collapse of Complex Societies. Tainter looked at several societies that gradually arrived at a level of remarkable sophistication then suddenly collapsed: the Romans, the Lowlands Maya, the inhabitants of Chaco canyon. Every one of those groups had rich traditions, complex social structures, advanced technology, but despite their sophistication, they collapsed, impoverishing and scattering their citizens and leaving little but future archeological sites as evidence of previous greatness. Tainter asked himself whether there was some explanation common to these sudden dissolutions.

The answer he arrived at was that they hadn’t collapsed despite their cultural sophistication, they’d collapsed because of it. Subject to violent compression, Tainter’s story goes like this: a group of people, through a combination of social organization and environmental luck, finds itself with a surplus of resources. Managing this surplus makes society more complex—agriculture rewards mathematical skill, granaries require new forms of construction, and so on.

Early on, the marginal value of this complexity is positive—each additional bit of complexity more than pays for itself in improved output—but over time, the law of diminishing returns reduces the marginal value, until it disappears completely. At this point, any additional complexity is pure cost.

Tainter’s thesis is that when society’s elite members add one layer of bureaucracy or demand one tribute too many, they end up extracting all the value from their environment it is possible to extract and then some.

The ‘and them some’ is what causes the trouble. Complex societies collapse because, when some stress comes, those societies have become too inflexible to respond. In retrospect, this can seem mystifying. Why didn’t these societies just re-tool in less complex ways? The answer Tainter gives is the simplest one: When societies fail to respond to reduced circumstances through orderly downsizing, it isn’t because they don’t want to, it’s because they can’t.

In such systems, there is no way to make things a little bit simpler – the whole edifice becomes a huge, interlocking system not readily amenable to change. Tainter doesn’t regard the sudden decoherence of these societies as either a tragedy or a mistake—”[U]nder a situation of declining marginal returns collapse may be the most appropriate response”, to use his pitiless phrase. Furthermore, even when moderate adjustments could be made, they tend to be resisted, because any simplification discomfits elites.

When the value of complexity turns negative, a society plagued by an inability to react remains as complex as ever, right up to the moment where it becomes suddenly and dramatically simpler, which is to say right up to the moment of collapse. Collapse is simply the last remaining method of simplification.

[Linkpost] Teaching Paradox, Europa Univeralis IV, Part I: State of Play

The article also references/discusses Seeing Like a State, which has been somewhat popular in rationalist circles after the SSC book review.

GPT-3 Gems

Apparently we'll be able to build lots of drones.

Load More