[Experimenting with transposing a concept coined in one domain to another domain, perhaps not completely legibly to the entire intended audience.]
In Darwinian Populations and Natural Selection, Peter Godfrey-Smith introduces a bunch of useful concepts allowing us to think more clearly about evolution and its constitutive processes. One of those is the distinction between three types of reproducers: collective, simple, and scaffolded (introduced in chapter 5.1). The short explanation of those categories is as follows.
Simple reproducers are entities capable of self-sufficient reproduction that are not composed of entities that themselves are self-sufficient reproducers.
The paradigmatic example is a bacterium. Eukaryotic cells are less paradigmatic examples because it involves mitochondria, which themselves are in-betweenish cases between simple and scaffolded.
Collective reproducers are entities capable of self-sufficient reproduction that are composed of entities that themselves are self-sufficient reproducers (simple or collective).
The paradigmatic example is a multicellular organism (insofar as we take the individual eukaryotic cells to be simple reproducers, see the point above).
Scaffolded reproducers are entities that reproduce only by relying on the reproductive machinery that is not "their own".
Paradigmatic examples are genes and viruses (central Dawkinsian replicators). Less paradigmatic examples include mitochondria and plastids,[1] as well as memes.
As is clear from the bullet sub-points, the categories are not clear,[2] but they do give some interesting anchor points to think about varieties of reproduction.
What if we try to apply those concepts to agency? How well do they carry over?
The LessWrong sphere has a long history of interest in collective agency, e.g., here, here, and here (see also the subagents tag). Unless we assume some weirdly exotic metaphysics or a non-standard notion of subagent,[3] it cannot be subagents all the way down. So, recursing on the constitutive structure of a collective agent, we will at some point arrive at a "simple agent",[4] whatever that means.
That's collective and simple agency. As far as I can tell, the main context where something like "scaffolded agency" is discussed is LLM scaffolding: you use a language model as a "cognitive engine" powering a so-called AI agent, which, in turn, is supposed to do something, and use the power of this cognitive engine to figure out how to do that something.
(Note that this has some interesting parallels with Dawkins's concepts of a replicator and its vehicle.)
If a scaffolded reproducer is an entity that reproduces by means of using reproductive machinery that is not "its own", then an obvious definition of a scaffolded agent would be:
Scaffolded agents are entities that achieve their goals only by relying on the "agentic machinery" that is not "their own".
How far should we stretch this concept? If I am achieving my goals using a hammer, a car, or a laptop, does it make me a scaffolded agent?
Maybe. Kinda?
But the case of me looking for a hammer seems meaningfully different from a virus relying on the mercy of its environment to drift it towards the membrane of a cell amenable to being infected by it. The latter appears much more "parasitic", "essentially relying on stuff that is not it", not self-sufficient in a deeper sense. The virus has less "basic interface-ish stuff" that would allow it to flexibly interoperate with tooling/resources that can be leveraged for goal achievement. That is, I have highly dexterous hands and a brain that can come up with ideas on how to use those hands to acquire goal-achievement-useful stuff, like a hammer, and then use it for achieving a goal.
The difference between scaffolded and non-scaffolded agents might be that, if you draw a "natural boundary" around the latter, it's going to include some "flexible interfacing bits", that allow the agent to complete itself to pursue a goal from a wide distribution of possible goals. A scaffolded agent is different in that it's waiting for a specific sort of thing to slot into its holes.
One frame is that a scaffolded agent is kinda dual to an "agent with a goal-shaped hole",[5] an agent waiting for a specification of what it is to do, and when it receives it, it gets to do it. A scaffolded agent is a "goal with some incomplete stuff", including stuff that allows it to hook onto some specific machinery that allows it to start pursuing its goal.
Does such a thing "merit" being called an "agent"? The epithet of "scaffolded" when added to "agent" takes away some stuff that would make the "agent" "complete", as well as adds a bit of stuff that allows it to "complete" itself into a "more complete agent" (at least a "basic agent" in the sense that John uses this term here). So a "scaffolded agent" is "both more and less" than an "agent simpliciter" (simple or collective). A less committing term we could use would be "(scaffolded) optimizer", an entity that "achieves" some goal, despite [things that we'd intuitively label "obstacles" to this goal] happening (insofar as those obstacles fall within some "tolerance region" of the optimizer).
If we think of agency as consequentialist cognition and adopt a primarily-sequential view of human agency, then an example of a scaffolded agent would be sequentially the goals in a human mind, jumping in and out of the slot which allows them for consequentialist-style planning. We can analogize this with retroviruses which got incorporated into the human genome, or transposons.
The eukaryotic cell is a former collective, and one that still has some features of a collective. Mitochondria, in different organisms, are at various locations on a road between simple and scaffolded.
The families are described by sketching partially idealized possibilities, which actual cases exemplify to various degrees. The families are not intended to cover all possible cases. The aim is to isolate three ways in which reproductive relationships can be part of a Darwinian process, each with different roles.
and
Many actual cases fall outside and between these categories. We can distinguish two reasons for that. First, the categories are presented by sketching idealized possibilities. There are many real-world cases that do not exactly match any of them, but are much closer to one option than the others. Order is being imposed on an unmanageable menagerie, and this is being done in part via idealization. ( The phrase ‘‘herding cats,’’ used to describe tasks involving the management of wayward things, is especially appropriate here.) There are also cases that are ‘‘mixed’’ in a more important sense, because they are balanced between two categories, or are on a road from one to another. The eukaryotic cell is a former collective, and one that still has some features of a collective. Mitochondria, in different organisms, are at various locations on a road between simple and scaffolded.
A simple agent might just be a part that makes sense to model as agentic, but not that part's sub-parts. If we think of agency as a matter of degree, then we just go with some threshold of agency that is relevant for our needs. (All of this assumes that we care about the thing that the concept of "simple agency" is trying to point at, which we may not.)
Very cool. I’ve been thinking a little bit recently about doing evolutionary simulations of agents, “scaffolded agents” seem a bit like viruses and it’d be interesting to hypothesize what kinds of viruses could exist in the context of LLMs
[Experimenting with transposing a concept coined in one domain to another domain, perhaps not completely legibly to the entire intended audience.]
In Darwinian Populations and Natural Selection, Peter Godfrey-Smith introduces a bunch of useful concepts allowing us to think more clearly about evolution and its constitutive processes. One of those is the distinction between three types of reproducers: collective, simple, and scaffolded (introduced in chapter 5.1). The short explanation of those categories is as follows.
As is clear from the bullet sub-points, the categories are not clear,[2] but they do give some interesting anchor points to think about varieties of reproduction.
What if we try to apply those concepts to agency? How well do they carry over?
The LessWrong sphere has a long history of interest in collective agency, e.g., here, here, and here (see also the subagents tag). Unless we assume some weirdly exotic metaphysics or a non-standard notion of subagent,[3] it cannot be subagents all the way down. So, recursing on the constitutive structure of a collective agent, we will at some point arrive at a "simple agent",[4] whatever that means.
That's collective and simple agency. As far as I can tell, the main context where something like "scaffolded agency" is discussed is LLM scaffolding: you use a language model as a "cognitive engine" powering a so-called AI agent, which, in turn, is supposed to do something, and use the power of this cognitive engine to figure out how to do that something.
(Note that this has some interesting parallels with Dawkins's concepts of a replicator and its vehicle.)
If a scaffolded reproducer is an entity that reproduces by means of using reproductive machinery that is not "its own", then an obvious definition of a scaffolded agent would be:
How far should we stretch this concept? If I am achieving my goals using a hammer, a car, or a laptop, does it make me a scaffolded agent?
Maybe. Kinda?
But the case of me looking for a hammer seems meaningfully different from a virus relying on the mercy of its environment to drift it towards the membrane of a cell amenable to being infected by it. The latter appears much more "parasitic", "essentially relying on stuff that is not it", not self-sufficient in a deeper sense. The virus has less "basic interface-ish stuff" that would allow it to flexibly interoperate with tooling/resources that can be leveraged for goal achievement. That is, I have highly dexterous hands and a brain that can come up with ideas on how to use those hands to acquire goal-achievement-useful stuff, like a hammer, and then use it for achieving a goal.
The difference between scaffolded and non-scaffolded agents might be that, if you draw a "natural boundary" around the latter, it's going to include some "flexible interfacing bits", that allow the agent to complete itself to pursue a goal from a wide distribution of possible goals. A scaffolded agent is different in that it's waiting for a specific sort of thing to slot into its holes.
One frame is that a scaffolded agent is kinda dual to an "agent with a goal-shaped hole",[5] an agent waiting for a specification of what it is to do, and when it receives it, it gets to do it. A scaffolded agent is a "goal with some incomplete stuff", including stuff that allows it to hook onto some specific machinery that allows it to start pursuing its goal.
Does such a thing "merit" being called an "agent"? The epithet of "scaffolded" when added to "agent" takes away some stuff that would make the "agent" "complete", as well as adds a bit of stuff that allows it to "complete" itself into a "more complete agent" (at least a "basic agent" in the sense that John uses this term here). So a "scaffolded agent" is "both more and less" than an "agent simpliciter" (simple or collective). A less committing term we could use would be "(scaffolded) optimizer", an entity that "achieves" some goal, despite [things that we'd intuitively label "obstacles" to this goal] happening (insofar as those obstacles fall within some "tolerance region" of the optimizer).
If we think of agency as consequentialist cognition and adopt a primarily-sequential view of human agency, then an example of a scaffolded agent would be sequentially the goals in a human mind, jumping in and out of the slot which allows them for consequentialist-style planning. We can analogize this with retroviruses which got incorporated into the human genome, or transposons.
Godfrey-Smith writes:
Godfrey-Smith writes:
and
E.g., subagency amounting to managed instrumentality, and then we can have a (semi-)symbiotic subagents, like in here.
A simple agent might just be a part that makes sense to model as agentic, but not that part's sub-parts. If we think of agency as a matter of degree, then we just go with some threshold of agency that is relevant for our needs. (All of this assumes that we care about the thing that the concept of "simple agency" is trying to point at, which we may not.)
See: What, if not agency? and Telopheme, telophore, telotect.