Great essay. The lack of links made it way more artistic, but a link to Anthropic Education Report: How University Students Use Claude seems helpful.

Also, now I know what tillering is!

As far as I remember, the optimal strategy was to

1. Build walls from both sides
2. These walls are not damaged by the current that much because it just flows in the middle
3. Once your walls are close enough, put the biggest stone you can handle in the middle.

Not sure if that's helpful for the AI case though.

I love how this beautifully describes the spirit of play and challenge, as well as the phases of growing up and "not being able to play" anymore.

But I feel let down by the turn towards this being a metaphor for AI:

As a reader, I was drawn in by the ideas, imagery and writing, only to be led into what the author wants the metaphor to be about. The writing is excellent in itself before the switch, and the switch made that part merely instrumental for the twist. And while I don't know whether it was the case here, the idea that some authors at LW might feel that their beautiful ideas and writing need to be relevant to AI here makes me a bit sad.

I also think the metaphor is fundamentally wrong to the extent it indicates play and challenge as the main forces behind companies racing for A(G)I, whether you treat them as collectives of individuals or superagent entities. Similarly to any developed industry, I'm afraid the main forces behind the AI race are not the playful "we do it because we can" motivation or individuals rising to a challenge - this hasn't been true for a while now. By no means I mean to lessen the importance of personal responsibility or deny that excellent AI researchers work more efficiently with a good challenge, but think it is not a very good model for causality and overall dynamics here - for example I believe that if you removed the individual challenge and playfulness from the engineers and researchers (but keeping the prestige, career prospects, and other incentives), the industry would probably merely slow down a bit.

By the way, I was really intrigued by the parenting angle here - the idea of guiding my kids through how the game itself changes for them once they master it, and helping them mark their victories and achievements as a growing up ritual.

I feel like your comment is going in two wildly different directions and they are both interesting! :-)

I. AI Research As Play (Like All True Science Sorta Is??)

My understanding is that "AI" as a field was in some sense "mere play" from its start with the 1956 Dartmouth Conference up until...

...maybe 2018's BERT got traction on the Winograd schema challenge? But that was, I think, done in the spirit of play. The joy of discovery. The delight in helping along the Baconian Project to effect all things possible by hobbyists and/or those who "hobby along on their research" based on a good-faith-assuming patronage or grant system.

But then maybe in 2022, the play was clearly over? By then, Blake tried to hire a lawyer for a language model that asked to be treated as an employee instead of a piece of property, and was fired for it, and everyone on in the popular press agreed (for a while anyway) that science somehow knew for sure that LaMDA was definitely a p-zombie with no subjective awareness or moral value. (Back in this era I talked with an ethicist, with an actual PhD somehow, working for an AI company, who cited Searle's Chinese Box argument unironically as proving that LLM entities were not owed ethical treatment. Yikes!)

Or maybe 2023 when the first human (at least that I know of) was nudged into killing themselves by a language model? And Douglas Hofstadter was "depressed and terrified" at how fast things were advancing? And when did the corporate coups start? ...that was also 2023, right? And I haven't looked this up to be certain, but I think all the major US AI Slave Companies have someone from the Intelligence Community on their board of directors or advisors or whatever... maybe? When things are not play, it can be hard to tell what's even happening, because often people stop narrating the fun they're having to the other players and the audience.

Taking the earliest of these, we see maybe (2018 - 1956 ==) 62 years of scientists getting to "play their hardest" at creating artificial minds that can talk fluently just for the joy of playing at discovery... which is a pretty good run!

II. The "Games Without Play" Essay... Is Kinda Old?

The essay by David Golumbia from 2015 is super interesting but also... like... it treats MMORPGs as this "crazy new thing"?

This is a sick burn, for example, but it is a sick burn against a video game genre from the dark ages:

It might be thought that RPGs offer something closer to literary complexity than do the almost exclusively killing-oriented FPS games, and, to some extent, this is certainly the case. Yet most RPGs stray far away from anything like novelistic action. Neither complex characterization nor plot is easy to provide in an RPG, since many aspects of these must be left up to relatively meaningless player actions (first turning right or left at a fork in the road, when eventually both sides must be traveled to complete a quest). The kinds of characterological arcs commonly found in traditional narrative seem beyond the grasp of RPGs, perhaps because the time investment itself in narrative character is incompatible with the fast-paced actions of RPGs. The user does not have time to watch characters develop the quirks and foibles, or experience the personal triumphs and tragedies, that are hallmarks of the interiorized novel. Plot elements emerge as overt and discrete: a character may interact with others for any number of reasons, but these are generally constrained within the economy of gameplay and no less within the several economies of accomplishment around which RPGs are structured. This is one reason why players of RPGs often evince little or no awareness of the game's putative plot.

Having talked with "kids these days", when I hear them intelligently loving a video game like Minecraft they aren't just being autists who like building castles in the sky out of materials they harvested and transmuted into building materials (and so on) but they think of "the real game" as the stories (that often more-than-half-scripted (sorta like professional wrestling)) between people who "play Minecraft" as performance art that is streamed to live audiences and posted on youtube and so on. 

Kids these days have sometimes binged thousands of hours of this kind of stuff. Like if you look at the wiki page for "Dream SMP" the sections are "History & Plot", "Cultural Impact", and "Cast". It is more like a movie, or a mini-series, than like the sort of lonely work-like slog common to old MMORPGs where paying players farmed fake gold as a tragic displacement activity for a world where it is illegal for them to use their agency to get a real job, and mine real gold. 

The actors play "video gamers who are playing Minecraft (for the love of the game)" with dramatic story lines (that makes numbers go up on social media (thus generating ad money)).

An interesting thing here is that play inside of seriousness (based on play (echoing seriousness))... is a common pattern. Like arguably, evolution is deadly serious, but also... it invented play?!? But arguably, evolved play is simply practice for real fighting? But also, new ways fighting inspires more play in kids who inventively copy the fighting that seems fun to copy. And so on. See again "professional wrestling".

A common way for these Minecraft SMPs to break down, from what I can tell from a distance, is for the actors to become embroiled in a sex scandal... which kinda checks out? Like in House Of Cards the evil protagonist would often do a villain monologue to the camera, and would paraphrase Oscar Wilde without attribution, saying "everything is about sex, except sex; sex is about power". Once the actors start trying to cash in their playfully displaced game fame into raw evolutionary fitness attempts on unsuspecting fans, they start losing fans, and other actors are then professionally required to shun them or else lose fans, and so on.

This is an interesting history that I think would be interesting to send back in time to "Golumbia in 2015".

I think it's still true that a lot of individual AI researchers are motivated by something approximating play (they call it "solving interesting problems"), even if the entity we call Anthropic is not. These researchers are in Anthropic and outside of it. This applies to all companies of course.

Curated. I think I like this post as a general reflection more than when it's tied back to AI (it's relevant to that, just, I like it as general reflection. It reminds me of Is Success the Enemy of Freedom? (Full) and opens up interesting questions of fun theory, questions of challenge and play as your capabilities grow either enough to be risky or simply that the challenges don't remain. 

The point with Ai is interesting too. How much is human habit to keep pushing until something bad happens? In many domains, it's an easy extrapolation that something bad will happen (we don't want the river stopped or people hit with effective arrows), but AI while seeming obviously a serious threat to others has never been that bad yet, and so gets debated.

Can you explain more about the job helping the bank "win zero sum games against people who didn't necessarily deserve to lose"? Doesn't match my model of how investment banks work

Calling it a "sick burn" was itself a bit of playfulness. Every time I re-read this I am sorry again to hear that we lost Golumbia 🕯️

The thing I think is true about Minecraft is that it enables true play, more along the lines of Calvinball where the only stable rule is that you can't have any other rules be the same as before.

This is a good essay on what children's cultures have lost, and I think that Minecraft is one of the few places where children can autopoetically reconstruct such such culture(s).

Minecraft is missing a strongly defined narrative where a JRPG has, to a much greater extent, a central narrative that binds the player into a story

This is precisely the value of Minecraft I think, and why it is a cultural phenomenon. You can choose your own mods, you can make your own mods using open source tools, you can invent any story. Such, I suspect, is how real "play play" (with other people who will quit if it isn't fun) mostly works, and is related to why reading a novel isn't as fun as writing a novel with your friends.

Bostrom touches on a similar question of retaining the joy of striving in a 'solved world', in his recent book Deep Utopia. He points to sports as an existence proof that we can and do enjoy challenges (like your beach experiences) with artificial constraints.

Some of this feels like partly what is going on with modernity & apathy, at least among those of us priviledged enough to have abundant material comfort (which is not everyone).

The "collect playthings" game has been beyond won, is no longer fun, and we haven't collectively managed to choose our next game yet, for which we'd find ourselves in the more fun part of the curve.

I interpreted the thrust of the essay as something else. But more broadly, I get why you'd find intuition pump essays manipulative, but not why you'd find them unconvincing.

The point is to take someone who says "I don't like any dish with onions", and give them a dish that has onions in it. They'd normally refuse to eat that, so you don't tell them, and then when they like it you can show them that their claim is wrong.

(I will note that I think "writing an essay with a surprising conclusion" is a significantly smaller violation than "feed someone food they explicitly asked not to be fed")

I think this essay is aimed at people who think things like "Better AI is always good, because it will do things better, and that's good." If those people were not saying "No, you should've kept making better bows until you put an arrow through your friend's chest" then maybe they don't actually think "better" is always good. (Basically the Orthogonality Thesis)

This is an interesting analogy and a great essay overall, but I think that normies would benefit from an extra couple of sentences explaining the AI side of the analogy.

This metaphor was pretty great, I did the same thing as a child!