Context: Post #8 in my sequence of private Lightcone Infrastructure memos edited for public consumption.
When you finish something, you learn something about how you did that thing. When you finish many things at the same time, you do not get to apply the lessons you learned from each of those things to the others. This insight, turns out, was non-trivially a core cause of the industrial revolution.
The assembly line is one of the foundational technologies of modern manufacturing. In the platonically ideal assembly line the raw ingredients for exactly one item enter a factory on one end, and continuously move until they emerge as a fully assembled product at the other end (followed right by the second item, the third item, and so on). This platonic assembly line has indeed been basically achieved, even for some of humanity's most complicated artifacts. A Tesla factory converts a pile of unassembled aluminum and some specialized parts into a ready-to-ride car in almost exactly 10 hours, all on a continuously moving assembly line that snakes itself through the Gigafactory.
A smooth assembly line is also the sign of a perfectly calibrated process. We know that we are not spending too much time on any part of our assembly. The conveyor belt moves continuously, always at the same speed, calibrated to be exactly enough to complete the task. If for some reason the task takes longer, because e.g. a worker is worse than a previous worker, we notice immediately.
In contrast to all of this, stands some human instincts around efficiency. If instead of making one item each from start to finish, we could just process a big batch of dozens or hundreds or thousands of items, we could, it seems, be so much more efficient. This is usually a lie.
The ever changing parable of the students making pottery/paper-airplanes/etc. is usually invoked at this point. While the parable has undergone many variations, this story from Atomic Habits is as far as I know the original one:
ON THE FIRST day of class, Jerry Uelsmann, a professor at the University of Florida, divided his film photography students into two groups.
Everyone on the left side of the classroom, he explained, would be in the “quantity” group. They would be graded solely on the amount of work they produced. On the final day of class, he would tally the number of photos submitted by each student. One hundred photos would rate an A, ninety photos a B, eighty photos a C, and so on.
Meanwhile, everyone on the right side of the room would be in the “quality” group. They would be graded only on the excellence of their work. They would only need to produce one photo during the semester, but to get an A, it had to be a nearly perfect image.
At the end of the term, he was surprised to find that all the best photos were produced by the quantity group. During the semester, these students were busy taking photos, experimenting with composition and lighting, testing out various methods in the darkroom, and learning from their mistakes. In the process of creating hundreds of photos, they honed their skills. Meanwhile, the quality group sat around speculating about perfection. In the end, they had little to show for their efforts other than unverified theories and one mediocre photo.
While one might accept that the assembly line has been deeply transformative in manufacturing, it might be less clear to see how the same principles would affect the operations of something like software engineering, which is a good chunk of what we do. However, the same principles have also been the driver of a non-trivial fraction of modern software development progress.
In the dark old days of software engineering, software would be shipped in what they called "releases".
The lifecycle of a release would start by a bunch of managers coming together and making a big long list of features they think the software they are working on should have. This list would then be handed to a small set of lead engineers to transform into something they would call the "spec", usually at least hundreds of pages long. This spec would then be handed to a set of programmers to "implement". The resulting piece of software would then be handed to a set of testers to test. Then handed back to the programmers to fix. Then they would do a big pile of user-testing to get product feedback on the resulting software. This would then result in an additional list of features, which would be translated into a spec, which would be implemented, tested and fixed.
And then finally, after many months, or even years, the software would be burned on a CD, and then be shipped out to users.
Contrast this with the processes dominating modern software engineering. Everything is continuously deployed. A single engineer routinely goes from having an idea for a feature, to having it shipped to users within hours, not months. Every small code change gets shipped, immediately. We avoid shipping many things at once, since it will make it harder for us to roll them back. This is an application of the principle of single piece flow/small batches.
At the management level, the opposite of single-piece flow is usually called "waterfall planning". A waterfall planning process is structured into multiple distinct stages of product development where big batches of changes get combined, audited, reviewed, iterated on, and eventually, sometime down the road, shipped to users. The alternative to waterfall processes are often called "lean processes" (also the eponymous cause of "the Lean Startup" book title).
The application of the principle of single piece flow to new domains can often produce enormous efficiency gains. One domain stuck deeply in the old ways, for example, is architecture and construction. A building gets built, or renovated, in a set of discrete, long stages. First the client "figures out what they need", then an architect draws up the blueprints, then a planner reviews the blueprints, then a contractor builds the whole building, then an auditor reviews the construction.
This is complete madness. How are you supposed to know what you need in a building if you have never built any part of it? How are you supposed to know what materials to work with if you don't know how well the different materials will work for you?
Lighthaven was renovated drastically differently from basically all other buildings built or renovated in the Bay Area. During renovation we would aim to finish a single room before we started working on the next room. After every room we would review what worked, what didn't work, which parts took longer than expected, and which parts turned out to be surprisingly easy. Our contractors were not used to this. We needed to change a huge amount about how they operated, but I think there was no other way for Lighthaven to have successfully gotten built if not for this.
Much of Lightcone's work should aim to ship as continuously as possible, even if there is no clear precedent for what single piece flow would look like in that domain. To show what this thinking looks like in-progress:
The ideal workshop, when I hold this consideration in mind, is a series of rooms that each participant walks through over the course of a week, with each station teaching them something, and preparing them for future stations. Every daylight hour, a newly educated participant leaves the workshop, with another person right behind them, and another person entering right at the start.[1]
Every single participant would be an opportunity to learn for all future participants. We could calibrate the efficiency and difficulty of each station (if necessary adapting to the participant), and would notice immediately if something was going wrong.
Unfortunately cohort effects loom large, as the experience of learning alone is very different than learning together, and this appears to be a big obstacle to making this ideal workshop happen. But I still am thinking that maybe there is some way.
In many ways Inkhaven is an application of single piece flow to the act of writing. I do not believe intellectual progress must consist of long tomes that take months or years to write. Intellectual labor should aggregate minute-by-minute with revolutionary insights aggregating from hundreds of small changes. Publishing daily moves intellectual progress much closer to single piece flow.
For Lighthaven event rentals, the month-long lead and planning time also generates so much inefficiency. The ideal series of events would be created one piece at a time. Of course the obstacle lies in people's calendars and their plans, who need to control their schedule weeks and days out, which requires locking in much about each event, long before the previous one has completed.
The application of single piece flow is also one big reason for why Lightcone is a gaggle of generalists. Specialization often breeds waterfall habits. A gaggle of generalists can all focus their efforts together on shipping whatever needs to be shipped right now, until it is shipped, and then reorient.
A "workshop snake" as Justis affectionately named it while helping me edit this post