This is a linkpost for https://fulcruminc.substack.com/p/the-bitter-lesson-for-software
The rigid schemas, fixed integrations, and deterministic pipelines that defined classical software are a form of encoded structure, and agents are coming for them too.
This worries me. Software is useful precisely because it is rigid and deterministic. That's why it's able to implement guarantees such as "nobody can decrypt your encrypted files without presenting your key, regardless of how convincingly they talk to the PGP software," or "this foreign-key column only contains values that match a primary key in that other table; you cannot convince PostgreSQL that it's in its interest to violate the key constraint."
I agree this is worrying. I don't think the agentification of software is necessarily good: it will likely leave humans with a worse understanding of how the processes around them work, which is particularly bad as we move towards building ASI. I just think it's pretty likely to happen.
Software is made of information flows
Software encodes information flows. An ERP system, for instance, takes procurement and locks it into a specific sequence of purchase orders, approval routing, invoice matching, and payment release. Git takes multiple people changing code and imposes a protocol of branching, diffing, reviewing, and merging. By codifying these information flows, software says how things should happen; it makes patterns repeatable and enforceable by expressing them in deterministic code.
Software took over the world because we learned to express useful actions into information flows, and then to express these information flows into deterministic code. The actions we could express this way were, by definition, within the space of consistent logical operations on rigid data structures.
Agents, too, encode information flows. And while they do so through the same infrastructure of software — that is, code — they are able to create information flows that are far more flexible. For one, they’re able to execute on more open-ended commands. But perhaps more importantly, agents work with the natural complexity of real-world tasks, rather than requiring that complexity be compressed into rigid data structures first. They do this by drawing on both system-specific information and the generalized knowledge they’ve absorbed through pre-training.
Further, as instances of software, they benefit from its useful properties — rerunnability, testability, and scalability. The potential for AI to replace human work hinges on this increasing flexibility as well as the practical advantages that come with being software.
But before the competition between AIs and humans plays out, we argue that agents first compete with the deterministic structures humans have encoded in software.
Agent software eats classical software
The existence of general reasoners pushes us towards replacing structured flows in our software stack with more general agent flows. The space of classical software it makes sense to build is shrinking. More and more software will be better described as agent software - infrastructural backbone in code calling various arrangements of agents in pre-defined or newly synthesized flows.
Repeatable and easy-to-measure components like infrastructure, systems, low-level transports, etc. will remain in code, albeit code probably optimized by models. When the user edits a collaborative document, there’s no need for an intelligent intermediary to save their changes to the server. But for higher-level tasks, agent software will overtake classical software by virtue of its greater generality.
Agent software will eat the well-defined world of pre-AI software and spit it out softer. This softening will change how we answer questions. To be more concrete, consider the following examples:
- Research-grade web scraping. We could produce standard, reproducible social-science analysis pipelines. Such a pipeline would be fed a prompt for the kind of data agents should scrape, how that data should get coded, and output a pre-defined analysis. When put in an agent-software script, the pipeline could also be reused and adapted to a variety of contexts, like systematic reviews or interview thematic coding.
- Full-coverage feature testing. Instead of a static test suite, we could place agents into a testing pipeline that varies with the feature being pushed. Such agents would trace the happy path and, if necessary, test against new edge cases each time a feature gets pushed. Since the whole thing runs in a standard CI environment, it can be rerun on every push like any other build step.
The bitter lesson for software
In machine learning, the bitter lesson says that general methods which scale with computation reliably beat methods which encode human knowledge as structure. Every attempt to hand-craft domain expertise into a system — chess heuristics, grammar rules, hand-engineered features — eventually lost to a simpler, more general method given enough compute.
With the improving capabilities of coding agents, the bitter lesson is now impressing itself onto the world of software itself. The rigid schemas, fixed integrations, and deterministic pipelines that defined classical software are a form of encoded structure, and agents are coming for them too.
Agent capabilities today require a shift in how we think about building software. Instead of asking what structure a system needs, we should be asking where we’ve been forcing structure just because code demanded it. The boundaries between what needs to be rigid and what can be flexible, what is and isn’t possible, have moved. And they will keep moving.