This is a special post for quick takes by Richard_Ngo. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
An analogy that might be banal, but might be interesting:
One reason (the main reason?) that computers use discrete encodings is to make error correction easier. A continuous signal will gradually drift over time. Conversely, if the signal is frequently rounded to the nearest discrete value, then it might remain error-free for a long time. (I think this is also the reason why two most complicated biological information-processing systems use discrete encodings: DNA base pairs and neural spikes. EDIT: Neural spikes may seem continuous in the time dimension but the concept of "brain waves" makes me suspect that the time intervals between them are better understood as discrete.)
Separately, agents tend to define discrete boundaries around themselves—e.g. countries try to have sharp borders rather than fuzzy borders. One reason (the main reason?) is to make themselves easier to defend: with sharp borders there's a clear Schelling point for when to attack invaders. Without that, invaders might "drift in" over time.
The logistics of defending oneself vary by type of agent. For physical agents, perhaps fuzzy boundaries are just not possible to implement (e.g. humans need to literally hold the water inside us). However, many human groups (e.g. social classes) have initiation rituals which clearly demarcate who's in and who's out, even though in principle it'd be fairly easy for them to have a gradual/continuous metric of membership (like how many "points" members have gotten). We might be able to explain this as a way of giving them social defensibility.
What is an "agent" in your ontology?
In one convo (at ILIAD 1), you named situational awareness as the criterion for distinguishing a part of an agent as a subagent. Is this also an important factor for calling something an "agent" simpliciter?
2Richard_Ngo
One thing I had in an earlier draft of this shortform: the concept of "brain waves" makes me suspect that the timings of neural spikes are also best understood as discrete. But I don't know enough about how brain waves actually work (or what they even are) to say anything substantive here.