Nitpick: IDK but my impression is that in the Raptor example, your point about "simple core" substantively applies, but also much (most?) of the visible complexity difference isn't about the design being much simpler than someone could imagine at the beginning, but rather stuff like
We needed a bunch of sensors at the beginning, now we don't anymore.
and
The design stabilized, so a bunch of parts (that previously wanted to be separate so that they could more easily be rearranged) can now be fused into one part, so there's more stuff hidden on the inside / made more spatially compact / has fewer joints.
and
We proved out performance of various elements, so we need fewer redundancies.
This stuff has the flavor of optimization that usually should come after the core design is worked out.
Yep, I agree that is a non-trivial part of it as well. A lot of the complexity was part of the optimization process itself, which could eventually be pruned away as people understood the parameters of the problem more. I agree that's a pretty different thing.
Context: Every Sunday I write a mini-essay about an operating principle of Lightcone Infrastructure that I want to remind my team about. I've been doing this for about 3 months, so we have about 12 mini essays. This is the first in a sequence I will add to daily with slightly polished versions of these essays.
The first principle, and the one that stands before everything else, is to question the requirements.
Here's how Musk describes that principle:
Question every requirement. Each should come with the name of the person who made it. You should never accept a requirement that came from a department, such as legal ... you need to know the name of the real person who made the requirement.
Then, you should question it, no matter how smart that person is. Requirements from smart people are the most dangerous, because people are less likely to question them. Always do so, even if the requirement comes from me [Musk]. Then make the requirements less dumb.
Here's some of how I think about it: plans are made of smaller plans, inside their steps to achieve 'em. And smaller plans have lesser plans, and so ad infinitum.
But it's hard to get your subplans right up front.
There are a bunch of reasons for this: sometimes something looks like it might be easy from 1,000 feet, and a lot gnarlier at 10 feet. Sometimes, one of your subproblems looks like it has a standard solution, but that standard solution is built to solve a bunch of issues you don't care about. The point of questioning the requirement is to de-silo your subplan, and look at a larger scope, and use that to figure out a better plan.
Most work done in most organizations is work that is best done never at all. The core of most solutions turns out to be much more elegant and simple than anyone who worked on the first version of something was able to imagine:

Why do we have requirements at all? Of course, we have plans and subplans, but those don't constitute "requirements". As will be a recurring theme in these principles, the need for things like the "requirements", and the associated risks, rear their head when a project needs more than one person to be completed, and as such, when tasks need to be split up across many individuals, and often shudder hierarchies of individuals.
The best way I have found to set up a delegee to understand the goal of a task and to question the associated requirements, is to communicate two things at the same time:
Abstractly communicating goals is usually a doomed endeavor, so start with a concrete solution sketch. Then, taking the solution sketch as a starting point, explain what problems it is trying to solve. No plan survives contact with the enemy, but the initial plan is still usually the best tool you have for conveying your eventual goal.
As a delegee, question the requirements. Throw away the solution sketch you were given, and think about how you would go about this problem from first principles, in the context of its broader goal.
In conversation this looks like this:
Weekly memo: Robert's priority this week is to set ourselves up to use the caching tools we now have available on Next.js to make a bunch of our page responses faster
Robert: I question the requirement to use Next.js's caching tools for this. I agree we might want a caching layer, but their API is kind of horrendous and has
unstablein the name. IMO we should just set up our own Redis instance if we want to do more caching.Me: Oh, yeah, sure, that works as well. Actually... now that I am looking more into this, we maybe want to wait for a bit on the next major version release which does something totally different for caching.
Robert: I am not finding myself enthused about their proposed approach, but agree it looks better than running our own Redis. I'll see how fast I can upgrade us to the latest version.
Most requirements come in the form of uncommunicated assumptions, so the above conversational pattern often plays out at different levels of abstraction:
Weekly memo: Ronny's priority this week is to simplify our check-in system since it's been a giant pain for the last 3 events that we ran, and I think we could just roll something ourselves with the Unifi API and Airtable that's much smoother. Feel free to ask Rafe for help on the technical parts.
Ronny: Ok, but my current take is that we shouldn't have a check-in system at all. We made like $3k with individual room-bookings during the last few events, and that was already sacrificing a bunch of rooms that could have been session spaces. I think we might want to cut the number of rooms we make bookable by 60%, and then I don't think the effort of optimizing the room booking system is worth it.
Tomorrow: "Do not hand off what you cannot pick up"