Suppose you have an empty plot of land and want to build a house. To someone without a civil engineering background, it may be tempting to just start building right on top of the bare soil.

However, this isn't The Sims. If you build your house on top of bare soil, you will likely face serious problems within a few years.

Why? Well, the short answer is "differential movement". But to explain what that means, let me first take a step back.

The goal for structural engineers: when you put something on the ground it should not move. In everyday life this is easy; objects are small and time horizons are short. If I'm at the park and I put my phone on the ground next to a tree, the soil isn't going to shift and cause a massive crack in my phone. But if I build a 200,000 pound house directly on the ground and wait a few months, I might not be so lucky.

The first thing to understand is that the ground isn't a solid, contiguous piece of matter. It's actually more like this:

Or maybe this:

This was an aha moment for me. Before watching this video, which I'm basing a lot of this section of the post on, my naive model of the ground was more like this:

That model serves me well when I put my cell phone down on the ground at the park, but it wouldn't serve me well in my hypothetical future as a home builder. The whole is composed of parts, and those parts can move independently of one another.

So what, exactly, actually causes the movement of these soil particles? Well, the weight of the home is part of it, but in practice, a lot of the issue is related to water. Soil expands when wet and shrinks when dry. Freeze-thaw cycles are especially bad. Tree roots can be a pain.

Anyway, why is this soil movement a problem? Well, take a look at this:

I don't know how to explain it well, but such differential movement can cause cracks.

To mitigate these risks, instead of building houses directly on soil, we build them on foundations. Something like this:

Or this:

When the soil moves beneath the foundation, the foundation will hopefully remain in tact. This allows us to accomplish our goal: when you put something on the ground it should not move.

Of course, even with a foundation, your luck will only last so long. Father Time and Mother Nature are a formidable duo.

If you want your foundation to last you 500 or 1,000 years instead of 75 years, that's probably doable — you can build something deeper and more solid — it's just going to cost you. Like most things, you'll need to make decisions about trade-offs. These decisions will depend on considerations like project size, time horizons, and budget.

Software engineering

What about software engineering? What would it mean to start building on top of "bare soil"? What sorts of "cracks" might form? After how long? How damaging would they be? What would a proper foundation look like? Which types of foundations are appropriate for which types of projects? And how long can a foundation realistically last?

As the idea for this post was brewing in my mind, I was hoping to discuss all of these questions in depth and say a bunch of smart and insightful things about them. But to be honest, that would be a bit beyond my pay grade. I'm a solid software engineer but I'm no John Carmack.

So then, I am pivoting to a more How To Write Quickly While Maintaining Epistemic Rigor type of approach. I'll describe some thoughts I have on some of these questions, and I'll leave it to people in the comments to continue the conversation. I know a lot of people here have software engineering backgrounds.

• Automated tests feel like something that provides a strong foundation. Without them it's easy to introduce bugs. I guess these bugs can be seen as "cracks", the problematic code would be the differential movement, and the automated tests would be the foundation that prevents the differential movement from actually causing a crack to form.
• Good logging also feels like something that provides a strong foundation. With good logging, when someone reports a bug, your logs often make it a whole lot easier to figure out the issue and address it. They also let you notice issues that don't get reported. They even help you detect things that are sketchy but haven't caused an issue yet. I have warning level logs in mind.
  
  Logs stretch this analogy a bit though: they're more about fixing cracks than preventing them. Although I guess they can also help you prevent cracks from becoming worse, which you could argue is sometimes closer in thingspace to "prevention" than "correction".
• When code quality is poor I get nervous when I am changing functionality or building new features. I worry about introducing bugs. I feel like this maps to a weak foundation. When you have a weak foundation, changing or extending the structure that is on top of the foundation is nerve-wracking.
• I'm having an issue in one of my projects right now where I have to run each of my automated tests individually. So like I'll have to run a command to run the tests I have for the log in page, then I have to run a second command to run the tests I have for the log out button, then a third command for the password reset functionality, etc.
  
  This really slows me down. But is it a gap in the foundation or is it something else? I'm not sure. Maybe we can count low velocity as a crack and say that a strong foundation prevents this low velocity crack from forming.
• What about types? All else equal, does a TypeScript codebase have a stronger foundation than a JavaScript codebase? Well, I guess that depends things like whether you think types reduce bugs and improve velocity. Personally, I was a late adopter, but I'm on the bandwagon of thinking types are important now.
• A foundation is an investment. How much you invest in a foundation depends a lot on the project. If you're building some sort of MVP or proof of concept, it probably isn't worth investing too much in the foundation. After all, if the idea is a failure and you ditch it after three months, the foundation you invested in won't be paying any dividends.
• The estimated lifespan of the project isn't the only thing that matters though. The cost of failure matters too. Especially when you're playing repeated rather than one-shot games. If you're dealing with sensitive financial or medical data it might be worth investing in a strong foundation even if the project doesn't have a long estimated lifespan.
• The "weight" and "flexibility" of the "structure" is also an important consideration. A "heavy" and "rigid" structure is more likely to form cracks than a lighter and more flexible one. In the domain of software, I guess these concepts map roughly onto the idea of "complexity", but I'm not really sure. I feel like a more academic person would be able to expand on this.
• "Softer" things like motivation and enjoyment matter too. If "overbuilding" the foundation sparks a lot of joy and makes you more eager to build your structure, it might be worth it to "overbuild".