The concept of scaffolded agent seems interesting to explore. Some thoughts:
1. This might be obvious from what you wrote and I'm missing it, but I would write the TL;DR as "A scaffolded reproductor contains (reproductive) information and uses external (reproductive) machinery; can we think of goal-y information using external goal-y machinery?".
2. With this TL;DR, it seems clear, as you yourself note, that the human using a hammer is not a good example, because the hammer isn't really goal-y machinery.
3. I also think that the final paragraph points at a different phenomenon: a decomposition of one agent into its goals and its internal goal-y machinery. This seems analogous to decomposing a human into DNA and ribosomes, and not similar to how a virus kidnaps those ribosomes so they are used with new information.
4. Some potentially better examples that I could think of:
a) A human boarding a ship, if we consider the ship as an agent acting in the universe to reach its goals. The human has a goal (which happens to be the same destination), and enters the "agency machinery" of the ship.
b) A better variation of this could be a human hitchhiking several rides to get to a destination. In this case, their goal doesn't coincide with any of the external agents' goals, so their goal gets fulfilled by many external goals being fulfilled, which might be closer to how a virus gets reproduced in addition to the normal reproduction of the host.
c) A different example of scaffolded agency could be a BookCrossing book. The book has the goal of being read and shared and tracked. Unlike the previous example, this goal isn't just a linear combination of external goals; it's something new.
5. Thinking about the last example makes me think: Could scaffolded agency be just... Dennett's design stance? Or more specifically, an affordance? Isn't a door-handle a scaffolded agent, waiting for external agency machinery to fulfill its goal of opening the door? If the answer is no, I think that we need to be looking for something that looks like an affordance, but doesn't have a designer, possibly something like the way a banana perfectly fits in a human hand?
Very cool. I’ve been thinking a little bit recently about doing evolutionary simulations of agents, “scaffolded agents” seem a bit like viruses and it’d be interesting to hypothesize what kinds of viruses could exist in the context of LLMs
[Experimenting with transposing a concept coined in one domain to another domain, perhaps not completely legibly to the entire intended audience.]
In Darwinian Populations and Natural Selection, Peter Godfrey-Smith introduces a bunch of useful concepts allowing us to think more clearly about evolution and its constitutive processes. One of those is the distinction between three types of reproducers: collective, simple, and scaffolded (introduced in chapter 5.1). The short explanation of those categories is as follows.
As is clear from the bullet sub-points, the categories are not clear,[2] but they do give some interesting anchor points to think about varieties of reproduction.
What if we try to apply those concepts to agency? How well do they carry over?
The LessWrong sphere has a long history of interest in collective agency, e.g., here, here, and here (see also the subagents tag). Unless we assume some weirdly exotic metaphysics or a non-standard notion of subagent,[3] it cannot be subagents all the way down. So, recursing on the constitutive structure of a collective agent, we will at some point arrive at a "simple agent",[4] whatever that means.
That's collective and simple agency. As far as I can tell, the main context where something like "scaffolded agency" is discussed is LLM scaffolding: you use a language model as a "cognitive engine" powering a so-called AI agent, which, in turn, is supposed to do something, and use the power of this cognitive engine to figure out how to do that something.
(Note that this has some interesting parallels with Dawkins's concepts of a replicator and its vehicle.)
If a scaffolded reproducer is an entity that reproduces by means of using reproductive machinery that is not "its own", then an obvious definition of a scaffolded agent would be:
How far should we stretch this concept? If I am achieving my goals using a hammer, a car, or a laptop, does it make me a scaffolded agent?
Maybe. Kinda?
But the case of me looking for a hammer seems meaningfully different from a virus relying on the mercy of its environment to drift it towards the membrane of a cell amenable to being infected by it. The latter appears much more "parasitic", "essentially relying on stuff that is not it", not self-sufficient in a deeper sense. The virus has less "basic interface-ish stuff" that would allow it to flexibly interoperate with tooling/resources that can be leveraged for goal achievement. That is, I have highly dexterous hands and a brain that can come up with ideas on how to use those hands to acquire goal-achievement-useful stuff, like a hammer, and then use it for achieving a goal.
The difference between scaffolded and non-scaffolded agents might be that, if you draw a "natural boundary" around the latter, it's going to include some "flexible interfacing bits", that allow the agent to complete itself to pursue a goal from a wide distribution of possible goals. A scaffolded agent is different in that it's waiting for a specific sort of thing to slot into its holes.
One frame is that a scaffolded agent is kinda dual to an "agent with a goal-shaped hole",[5] an agent waiting for a specification of what it is to do, and when it receives it, it gets to do it. A scaffolded agent is a "goal with some incomplete stuff", including stuff that allows it to hook onto some specific machinery that allows it to start pursuing its goal.
Does such a thing "merit" being called an "agent"? The epithet of "scaffolded" when added to "agent" takes away some stuff that would make the "agent" "complete", as well as adds a bit of stuff that allows it to "complete" itself into a "more complete agent" (at least a "basic agent" in the sense that John uses this term here). So a "scaffolded agent" is "both more and less" than an "agent simpliciter" (simple or collective). A less committing term we could use would be "(scaffolded) optimizer", an entity that "achieves" some goal, despite [things that we'd intuitively label "obstacles" to this goal] happening (insofar as those obstacles fall within some "tolerance region" of the optimizer).
If we think of agency as consequentialist cognition and adopt a primarily-sequential view of human agency, then an example of a scaffolded agent would be sequentially the goals in a human mind, jumping in and out of the slot which allows them for consequentialist-style planning. We can analogize this with retroviruses which got incorporated into the human genome, or transposons.
Godfrey-Smith writes:
Godfrey-Smith writes:
and
E.g., subagency amounting to managed instrumentality, and then we can have a (semi-)symbiotic subagents, like in here.
A simple agent might just be a part that makes sense to model as agentic, but not that part's sub-parts. If we think of agency as a matter of degree, then we just go with some threshold of agency that is relevant for our needs. (All of this assumes that we care about the thing that the concept of "simple agency" is trying to point at, which we may not.)
See: What, if not agency? and Telopheme, telophore, telotect.