This post reminded me of another great post that delves more deeply into the S-curve phenomenon, Invisible Asymptotes. It's written by someone who was involved with Amazon in their early days, as they found their growth slowing and needed to predict what was the next bottleneck to their growth as they went through the S curve cycle.
For me, in strategic planning, the question [ was to flush out what I call the invisible asymptote: a ceiling that our growth curve would bump its head against if we continued down our current path. It's an important concept to understand for many people in a company, whether a CEO, a product person, or, as I was back then, a planner in finance.
Amazon's invisible asymptote
Fortunately for Amazon, and perhaps critical to much of its growth over the years, perhaps the single most important asymptote was one we identified very early on. Where our growth would flatten if we did not change our path was, in large part, due to this single factor.
We had two ways we were able to flush out this enemy. For people who did shop with us, we had, for some time, a pop-up survey that would appear right after you'd placed your order, at the end of the shopping cart process. It was a single question, asking why you didn't purchase more often from Amazon. For people who'd never shopped with Amazon, we had a third party firm conduct a market research survey where we'd ask those people why they did not shop from Amazon.
Both converged, without any ambiguity, on one factor. You don't even need to rewind to that time to remember what that factor is because I suspect it's the same asymptote governing e-commerce and many other related businesses today.
Shipping fees.
People hate paying for shipping. They despise it. It may sound banal, even self-evident, but understanding that was, I'm convinced, so critical to much of how we unlocked growth at Amazon over the years.
People don't just hate paying for shipping, they hate it to literally an irrational degree. We know this because our first attempt to address this was to show, in the shopping cart and checkout process, that even after paying shipping, customers were saving money over driving to their local bookstore to buy a book because, at the time, most Amazon customers did not have to pay sales tax. That wasn't even factoring in the cost of getting to the store, the depreciation costs on the car, and the value of their time.
People didn't care about this rational math. People, in general, are terrible at valuing their time, perhaps because for most people monetary compensation for one's time is so detached from the event of spending one's time. Most time we spend isn't like deliberate practice, with immediate feedback.
Wealthy people tend to receive a much more direct and immediate payoff for their time which is why they tend to be better about valuing it. This is why the first thing that most ultra-wealthy people I know do upon becoming ultra-wealthy is to hire a driver and start to fly private. For most normal people, the opportunity cost of their time is far more difficult to ascertain moment to moment.
You can't imagine what a relief it is to have a single overarching obstacle to focus on as a product person. It's the same for anyone trying to solve a problem. Half the comfort of diets that promise huge weight loss in exchange for cutting out sugar or carbs or whatever is feeling like there's a really simple solution or answer to a hitherto intractable, multi-dimensional problem.
Solving people's distaste for paying shipping fees became a multi-year effort at Amazon. Our next crack at this was Super Saver Shipping: if you placed an order of $25 or more of qualified items, which included mostly products in stock at Amazon, you'd receive free standard shipping.
The problem with this program, of course, was that it caused customers to reduce their order frequency, waiting until their orders qualified for the free shipping. In select cases, forcing customers to minimize consumption of your product-service is the right long-term strategy, but this wasn't one of those.
That brings us to Amazon Prime. This is a good time to point out that shipping physical goods isn't free. Again, self-evident, but it meant that modeling Amazon Prime could lead to widely diverging financial outcomes depending on what you thought it would do to the demand curve and average order composition.
To his credit, Jeff decided to forego testing and just go for it. It's not so uncommon in technology to focus on growth to the exclusion of all other things and then solve for monetization in the long run, but it's easier to do so for a social network than a retail business with real unit economics. The more you sell, the more you lose is not and has never been a sustainable business model (people confuse this for Amazon's business model all the time, and still do, which ¯\_(ツ)_/¯).
The rest, of course, is history. Or at least near-term history. It turns out that you can have people pre-pay for shipping through a program like Prime and they're incredibly happy to make the trade. And yes, on some orders, and for some customers, the financial trade may be a lossy one for the business, but on net, the dramatic shift in the demand curve is stunning and game-changing.
The article is quite long (this lengthy excerpt is, like, just the prologue). The post covers a number of interesting areas.
(Overall this blog actually makes me feel the most like reading a Scott Alexander post, i.e. most posts are quite long, covering multiple lenses through which to look at an interesting problem)
I wanted to work in this idea of getting ahead of constraints but it didn't really seem to fit anywhere.
I agree Eugene is among my top 3 business strategists to follow online and I learned a lot about how to think about strategy by reading his blog.
Idea - a thread for the best blogs to follow on every subject.
My comment says it all:
Intriguingly, after coming back to this comment after only 3 months, my feeling is something like "That seems pretty obvious. It's weird that that seemed like a new insight to me."
So I guess you actually taught me something in a way that stuck.
Thanks again.
I'm glad to have read this. In particular:
Sometimes, people get confused and call S-curves exponential growth. This isn't necessarily wrong but it can confuse their thinking. They forget that constraints exist and think that there will be exponential growth forever. When slowdowns happen, they think that it's the end of the growth - instead of considering that it may simply be another constraint and the start of another S-Curve.
This is obvious in hindsight, but I hadn't put my finger on it.
Thank you!
Intriguingly, after coming back to this comment after only 3 months, my feeling is something like "That seems pretty obvious. It's weird that that seemed like a new insight to me."
So I guess you actually taught me something in a way that stuck.
Thanks again.
Plus one on that point.
In once sense though it seems a rejection of, what I will call, the S-Curve mentality. That would be the thinking all growth always plateaus (and it seems that is a dominant view in terms of economic growth there -- developing economies can grow faster then developed economies so all these fast growers are doomed to the fate of Japan, Europe or the USA). That thinking can lead to acceptance rather than effort to overcome some current limitation/constraint.
Nominated. S-Curves have remained important to my understanding of the world. (I'm not sure I've consciously thought about them as often as I should have, but I think I've thought about them non-zero. This post still seems like a good introduction to them)
How do I know when some trend isn't made of S-curves? How do S-curves help me make predictions, or, alternately, tell me when I shouldn't try predicting? Is this falsifiable?
> Is this falsifiable?
Innovation research is notoriously hard to falsify and subject to just-so stories and post-hoc justifications.
One of the things I find compelling about S-curves is just how frequently they show up in innovation research coming from different angles and using different methodologies.
Some examples:
> How do S-curves help me make predictions, or, alternately, tell me when I shouldn't try predicting?
By understanding the separate patterns, they can give you an idea of the most likely future of different technologies. For instance, here's a question on LW that I was able to better understand and predict because of my understanding of S-curves and how innovations stack.
> How do I know when some trend isn't made of S-curves?
I think understanding how to work with fake frameworks is a key skill here. Something like S-curves isn't used in a proof to get to the right answer. Rather, you can use it as evidence pointing you towards certain conclusions. You know that they tend to apply in an environment with self-reinforcing positive feedback loops and constraints on those feedback loops. You know they tend to apply for diffusion and innovation. When things have more of these features, you can expect them to be more useful. When things have less of these features, you can expect them to be less useful. By holding up a situation to lots of your fake frameworks, and seeing how much each applies, you can "run the Bayesian Gauntlet" and decide how much probability mass to put on different predictions.
I think you'll always be working in S-curves if you're in a finite system. The trick is to be able to detect the rate-limiting factor. That's the factor that marks the inflection point between exponential growth and the beginning of the slowdown. For classic examples like bacterial growth that might be nutrients, space, elimination of waste, etc.
The hard part is determining whether you've considered all the rate-limiting factors involved. Going back to bacterial growth, if you think food is the rate-limiting factor and you predict your culture will continue to grow for six hours, you might be surprised when you hit an inflection point after three hours because waste products start killing bacteria off. This same principle can be applied in technology and elsewhere, where people often aren't even looking for rate-limiting factors and appear to assume exponential growth in a finite system.
Agree, recognizing constraints before they limit you is key if you're looking to create growth (or alternatively, adding multiple constraints if you're looking to slow growth).
If you want to slow growth, pick any limiting factor and apply pressure. One will do.
Sometimes a trend continues growing exponentially for a long time before bumping up against a limiting factor. The thing to remember about an S-curve is that if you plot it on a log scale the first half of the curve looks like a straight line all the way backward. That's because it's exponential growth at the beginning, so every new observation dwarfs all those that came before. Sometimes we spend a lot of time in exponential growth phase and people write articles about how it'll go on forever, and The Singularity, and whatnot. When you don't know the limiting factor, it's very tempting to fit your model to exponential growth, only to get burned later on.
Agree, this misconception (and seeing it everywhere) is one of the things that made me wrote the article (particularly the part about "exponentional growth vs. s-curves).
The other side of it is when people think that trends are made of a single s-curve, and think that when growth is slowing down, that means that the trend is done forever, rather than simply the start of another s-curve when the constraint is defeated.
Yes, I think it's an excellent article, especially the observation about constraints. If we can correctly identify which elements are constraining a system we have a path to return to exponential growth.
Still, we'll see articles lamenting that "despite how we've overcome Constraint X, growth hasn't returned." The world is multi-causal/multi-factoral, though. More than one factor can constrain growth. It is often an engineering problem, and focusing on the system as driven by rationally understandable forces is important. Otherwise the default seems to be to view trends as 'magical growth' and make illogical predictions based on that thinking.
In the case of growth in the computer hardware industry, where you have a veritable army of engineers focused on the problem, is it any wonder we continuously overcome constraints?
I think I may have missed this post when it came out, because apparently I didn't update it, and now that I have some experience with forecasting it's mostly old news to me -- but damn this post is a gem. I wish I had read it ten years ago. Everyone interested in tech forecasting should read it.
I've known about S-curves for a long time, and I don't think I read this the first time. If you don't know S-curves exist, this has good info, and it seems to be well explained. There are also a few useful nuggets otherwise. As someone who has long known of S-curves, hard to say how big an insight this is to others, but my instinct is that while I have nothing against this post and I'm very glad it exists, this isn't sufficiently essential to justify including.
This was a great explanatory post, that distilled a lot of complex ideas into something short and accessible. I've curated it.
A minor thing that would have improved it is, for the various types of curves (especially ones named after people), it might have been nice to link to an existing canonical reference for that idea, that covers it in more depth or at least gives a sense of how other people have thought about it.
Scalable systems all turn out to have these S-curve dynamics. It's roughly based on how networks operate, and other people have expounded upon this extensively – e.g. Geoffrey West, etc. Networks have certain identifiable elements, which can be projected onto specific situations for a deeper analysis. You can read a bit more about the System model and how it impacts technology innovation here: https://www.amazon.com/Scalable-Innovation-Eugene-Shteyn/dp/1466590971
Changes I made based on the yearly review:
Update: Made these changes
I originally wrote this post because I saw quite a few of what I perceived mistakes in the reasoning of rationalists around predicting trends and innovation.
At the time, I thought that simply making a post explaining the models they were missing would create a resource to help correct these errors.
I now think this was a bit naive. New models don't automatically propagate to old believes, and I believe this post would be better served by giving specific examples of where reasoning can be confused or incomplete without these models.
Indeed, the one part of the post where I do this is in fact the part that had the most impact.
I agree with Zvi that as the post stands doesn't do this well enough to merit inclusion in the final collection, but that a version of the post that does could in fact be useful.
I also agree with Hamnox that there needs to be more clear resources, and the image links need to be fixed. I disagree about the use of evolution, which just means the study of change over time and is used in scientific disciplines like Cosmology. Eliezer is just wrong here.
Oh man I liked this post but I bet if you laid out the concrete examples of where this goes wrong, I'd learn something useful.
I do think this is a great standalone post, which has indeed affected my thinking at least somewhat.
Broken image link! Broken image link! Sad.
Using "evolution" when referring to things other than breeding populations, BOO!
Important concept. Very light on backing evidence and references. I wanna hear more about Systems and Networks angles. Could do with fewer examples of Innovation.
Epistemic Status: Innovation research and business research is notoriously low quality, and so all the ideas here should be viewed through that lense. What's impressive about the S-curve and evolution trends literature is how remarkably self-consistent it is using a wide variety of research methods. Whether Simon Wardley analyzing news article about different technologies, Clayton Christensen doing case studies of a specific industry, or Carlotta-Perez taking a historical approach of tracking different technologies, the same S-curve pattern and evolution trends seem to show up. This too should be taken into account when evaluating these ideas.
Basics
This is an S-curve.
The S-curve is a fundamental pattern that exists in many systems that have positive feedback loops and constraints. The curve speeds up due to the positive feedback loop, then slows down due to the constraints.
When the constraint is broken, the positive feedback loop ramps back up, until it hits another constraint.
Recommended Resource: Invisible Asymptotes, which gives a visceral feel for this process of positive feedback and constraints
Common Mistake: Confusing S-Curves With Exponential Growth
Sometimes, people get confused and call S-curves exponential growth. This isn't necessarily wrong but it can confuse their thinking. They forget that constraints exist and think that there will be exponential growth forever. When slowdowns happen, they think that it's the end of the growth - instead of considering that it may simply be another constraint and the start of another S-Curve. Knowledge of overlapping S-Curves can help you model these situations in a more sophisticated way.
Diffusion S-Curves
The S-curve pattern is quite common in the spread of ideas, practices, and technologies, although it rarely looks quite as pretty. The example below shows "diffusion s-curves" - How a technology spreads through a population (in this case US households
The positive feedback loop in this case is word of mouth, and the constraints represent fundamental barriers to certain market segments or growth such as simplicity, usability, scalability, price, etc.
This creates smaller s-curves around adoption among specific market segments, and larger s-curves that represent the overall market penetration of the idea, practice, or technology.
Recommended Resource: Wikipedia on Diffusion of Innovation
Evolution S-Curves
In addition to Diffusion S-curves in technology, ideas, and practices, there are Evolution S-Curves. These represent the increase in the traits of these ideas that make them usable in more situations and desirable for more people. When you break through a constraint in one of these properties through innovation, this can often coincide with "unlocking" a new diffusion curve by opening up a new market that wouldn't previously have used your technology or idea.
In this case the positive feedback loop is the increased understanding and expertise that comes from diffusion of a new innovation in your idea or technology, and the constraint represents fundamental assumptions in the idea, practice, or technology that must be changed through another innovation to make the idea, practice, or technology more desirable.
In the example below the desirable property is hardware speed. Fundamental leaps are made to break through a speed constraint, and then iterated on through the positive feedback loop of information and expertise increasing from adoption. This hits diminishing returns as the new innovation is optimized, and then a new fundamental innovation is needed to overcome the next constraint.
Recommended Resource: Open University on Evolution S-Curves
Common Mistake: Confusing Diffusion S-Curves with Evolution S-Curves
Sometimes, I see people make the mistake of assuming that evolution and diffusion s-curves follow the same cycle. Most often, the mistake made here is assuming that when a particular innovation has saturated a certain market, that also means it has "reached its final form" and has no more evolving to do.
There is a related truth - often, an innovation becoming more diffuse will drive innovation as new use cases become apparent. And vice versa, often new innovations will open a new market up by creating use cases that were previously impossible.
However, the two types of curves are driven by two different feedback loops and two different constraints. There's no reason to expect that they will follow each other, and no reason to expect that one curve leveling off will cause the other curve to level off.
S-Curves Patterns
S-curves become quite useful when paired with an understanding of evolutionary patterns. They can allow you to see in a broad sense what's coming next for an idea, practice or technology. They can prevent surprises and give you a tool to stay ahead of changes.
There are patterns that exist for both diffusion and evolution S-curves.
Diffusion Patterns
Diffusion patterns describe common themes that happen as trends diffuse through a population. They apply on the micro-level to individual population-segments, and on a macro-level to the overall population.
Diffusion of Innovation
The diffusion of innovation describes 5 separate stages of a diffusion curve: Innovators, Early Adopters ,Early Majority, Late Majority, and Laggards. By understanding the traits of each of these groups, you can get a broad idea of what to expect, and how to slow or speed up adoption.
Recommended Resource: Diffusion of Innovations book by Everett Rogers
The Chasm
The Chasm describes a common constraint that occurs in a market segment between "early adopters" - who are willing to put up with a lot, and "early majority", who expect a lot. There is often a number of evolutionary constraints that must be broken through to bridge this single diffusion constraint and many new ideas, practices, and technologies get stuck in the chasm for that reason.
Recommended Resource: Crossing the Chasm book by Geoffrey Moore
Common Mistake: Assuming a Technology is Irrelevant Because it's Only Useful for a Small Group
A common mistake that I see is assuming a technology won't have a broader relevance, and using as evidence that it's only used by a small group of relatively abnormal people.
Now, what is true is that not all technologies eventually get adopted by everybody, some stay relatively niche. But it's not very good Bayesian evidence to say that because a technology is used by a small group of weird people, it will not have a broader impact. These diffusion patterns tell us that in fact that MOST technologies that eventually get widespread adoption go through this phase.
Furthermore, they tell us that many of those technologies often get stuck for a while at this early stage because of the Chasm. So even if a technology has staid at this tage for a while (e.g. cryptocurrency), it's still very little evidence towards that technology not being lifechanging in the future. (In contrast, a technology stalling for a long time at some point past the chasm is better evidence that it may have reached saturation)
Evolution Patterns
Evolution patterns describe common ways that innovations evolve over time to become increasingly desirable. They apply on the micro-level to individual innovations within a trend, and on a macro-level to the evolution of trend as a whole.
Wardley Evolution
Innovations tend to go through four stages - the initial prototype, custom built versions, productized versions that compete, than comoditized versions that are all basically the same. By understanding where you are, you can understand the type of competition likely to happen, the types of processes likely to yield improvements, and large changes that will be needed to stick with the market.
Recommended Resource: Learn Wardley Mapping- Free Resource from Ben Mosior
Common Mistake: Not reasoning about likely changes in how the market will be structured.
A common mistake I see when people reason about the future of e.g. Machine Learning, is that they reason as if the current economic style (how people make money from machine learning) will continue the way it has been.
What Wardley Evolution tells us is rather that it's very frequent for the way a market charges for and makes money with a particular innovation changes, and that change tends to fairly predictable.
For instance, I've seen analysis of Machine learning that assumes it will continue to be productized (which leads to very different dynamics in terms of competitive landscape and strategy between different AI vendors), rather than recognizing that it will eventually be commoditized and become a utility.
Simplicity - Complexity - Simplicity
Innovations tend to start out relatively simple as a new approach to a problem. They become increasingly complex to cover more use cases and be more robust, and then become simple again as refinements are made and they're distilled to their essence.
Recommended Resource: TRIZ for Dummies book by Lilly Haines-Gadd
Common Mistake: Assuming a Particular Innovation is a Dead End Because It's Gotten Too Complex
One mistake I see pretty frequently is people describing a particular innovation, and saying "well, we've added more and more complexity to this and it's gotten increasingly minimal returns so I expect there too not be too much more innovation in this area.
This can be true, but only if there are other indicators that this is already at the end of the innovation curve. Oftentimes, what's actually happened is that it's near the midpoint of it's innovation curve, and the next innovations will be around compressing/simplifying all the things that have been added. This simplification process then allows the innovation to be used a component to build further innovations off of, as it's simple enough to be commoditized.
Disruptive Innovation
Sometimes, innovations overshoot the mainstream populations needs on a particular dimension in order to be powerful for a particularly lucrative part of the population. In this case, these innovations or often overtaken by subsequent innovations that lower the performance on that dimension in order to raise it on other dimensions (example: Lower flexibility of a software product but raise the simplicity), these innovations can then "disrupt" the original innovation.
From the perspective a current innovation, the disruptive innovation appears to start below it in the s-curve, but it's able to gain adoption because the particular performance feature of that innovation is already higher than the market needs, and the new product competes on a different performance feature that is not even a target of.
Recommended Resource: The Innovator's Dillema - Book by Clayton Christensen
Common Mistake: Assuming a Particular Player will Win Because They're Big and Have Lots of Resources
One understandable assumption to make is that big players with more resources will always win. This isn't necessarily a bad assumption to make - disruptive innovations are much rarer than sustaining innovations.
However, having the disruptive innovation model can help you not make the mistake of just assuming that there's nothing that can topple the current champ - it gives you a clear model of exactly how this happens, and may even point out industries or areas where you're more likely to see this disruption take place.
Gartner Hype Cycle
The Gartner Hype Cycle describes a particular way that the media over-inflates people's expectations of new innovations in comparison to how evolved they actually are for a particular market segment's needs.
Recommended Resource: Mastering the Hype Cycle - Book by Jackie Fenn (Disclaimer: Haven't read this one, only aware of the Gartner Hype Cycle in passing)
Common Mistake: Discounting a Particular Technology Because it Was Overhyped in the Past
I've frequently seen arguments of the form - "Oh, you think this technology will have a massive impact? That's what they were saying a couple years ago and they massively overpromised.
Like other patterns, this is not saying that there aren't technologies that are massively overhyped and don't pan out. However, knowledge of the Gartner Hype cycle an show you that almost all popular technologies were once overhyped, so the argument of "this technology was overhyped in the past" isn't very good evidence of how transformative it will be. Rather, you'll want to map it against an evolution S-curve to see how overhyped you expect it to be relative to it's current level of evolution.
Windermere Buying Hierarchy
The Windermere Buying Hierarchy describes four different improvement focuses that an innovation optimizes over time. First, it's trying to solve for functionality, then reliability, then convenience, and finally price. This loosely maps to the stages of Wardley Evolution.
Recommended Resource: Haven't found a good one, learned about it through Clayton Christensen's work.
Common Mistake: Using Reliability, Convenience or Price as a Reason an Innovation Won't be Successful
You know the drill by now... it's not that reliability, convenience, or price are never reasons that a technology fails. But you'll want to map these against the evolution S-curves. It's common to see arguments about a technology not being viable because it's too expensive, when the S-curve is still WAYY at the early stage and we wouldn't even have expected the market to start thinknig about price optimization yet.
Only if the market has already reached that point in the S-curve and optimized that trait as much as it could, should you use this as a viable reason why you don't expect the technology to spread further.
Conclusion
S-curves and s-curve patterns are a useful tool for quickly analyzing systems, particularly when looking at diffusion of trends and evolution of innovations. They can heuristically identify solutions and probabilities that would otherwise be quite time consuming to figure out using something like a full system or functional analysis.
Hopefully you find this tool useful in your quest to understand all the things.