Lean Startup Reading Comprehension Quiz

I read/listened to  Lean Startup back in 2014. Reading it helped me realize many of the mistakes I had made in my previous startup, mistakes I made even though I thought I understood the "Lean startup" philosophy by osmosis.

Indeed, "Lean Startup" is a movement whose terminology has spread much faster than its content, creating a poisoned well that inoculates people against learning 

For example, the term "minimum viable product" has been mutated to have a meaning emphasizing the "minimum" over the "product," making it harder to spread the actual intended idea.  I blogged about this a long time ago: http://www.pathsensitive.com/2015/10/the-prototype-stereotype.html

Anyway, this post was a nice review! I had to guess on some of the questions, which is probably good; if I'm successful, it means I really internalized it. Thanks!

Lean Startup Reading Comprehension Quiz

1. What is the difference between learning and validated learning?

 Validated learning is learning that has been tested empirically against users/the marketplace.

2. True or false: "According to the author, a startup with exponential growth in metrics like revenue and number of customers is doing well." Explain your answer.

 False. This is only true if those metrics imply a path of long-term sustainable profitability. If the startup in question is Github, it probably does. If it's Groupon.....

3. Finish the sentence: "almost every lean startup technique we've discussed so far works its magic in two ways:"

  By reducing inventory and increasing validated learning. 

4. Ries argues that startups should pay more attention to innovation accounting than traditional accounting. Name two ways in which startups can change their financial metrics to accomplish innovation accounting.

 (a) Estimate the value of patents/trade secrets and track on an internal balance sheet. (b) Require VoI calculations and add such numbers to an internal balance sheet. 

5. Describe, concretely, what a car company's supply chain would look like if it used push vs pull inventory.

 Push: Each supplier pumps out parts, which are stockpiled in storerooms and warehouses.  Each factory will regularly, e.g.: be shipped all the stuff it needs for the next month. Pull: Each factory keeps just a few days of parts needed, places frequent orders for the next few day's worth.

6. Ries applies the pull inventory model to startups. But what is the unit that is being pulled, and where does it obtain the "pull signal"?

 The unit is "aspects of the business that deliver value to customers." The initial pull is validated market demand, which then translates to internal demand for features/process.

7. True or false: "Lean manufacturing is meant to give manufacturers an advantage in domains of extreme uncertainty". Explain your answer.

 True. Lean manufacturing allows manufacturers to retool and change their production much faster, greatly cheapening the cost of creating a suboptimal or unwanted product.

8. True or false: "Lean manufacturing is about harnessing the power of economies-of-scale."

 False. Lean manufacturing cheapens the cost of small runs, making the manufacturer more competitive at a lesser scale.

9. Ries discusses an anecdote of a family folding letters. The dad folds, stamps, and seals one letter at a time; whereas the kids begin by folding all letters, then stamping all, etc. Name two reasons Ries' considers the dad's method superior.

 a) Not having to manage the intermediate outputs. b) Can discover issues later in the pipeline earlier.

10. True or false: "A consequence of lean manufacturing is that the performance of each employee as an isolated unit, in terms of output per unit of time, might *decrease*." Explain your answer.

  Lean manufacturing comes with much higher switching costs. An employee's output might shrink, but more of it will go towards useful ends.

11. Give an example of what a “large batch death spiral” might look like in practice. 

 My game team is running behind schedule.  To catch up, I ask the artists to produce assets without waiting for them to be tested.  This then creates a large batch of work for the programmers to implement the graphics, which produces a large back of comments. This gets passed back to the artists, who do a huge number of revisions at once. The cycle continues.

12. According to Ries, the “Five why’s” method is a control system (though he doesn’t say so explicitly). What does it control, and how? 

 It puts a damper on major failures; it creates a mechanism by which a failure is turned into a systematic, mitigating change.

13. Explain the meaning of Toyota proverb “Stop production so that production never stops”

 Do regular maintenance and improvement work to prevent larger future problems.

Limiting Causality by Complexity Class

Causal inference has long been about how to take small assumptions about causality and turn them into big inferences about causality. It's very bad at getting causal knowledge from nothing. This has long been known.

For the first: Well, yep, that's why I said I was only 80% satisfied. 


For the second: I think you'll need to give a concrete example, with edges, probabilities, and functions. I'm not seeing how to apply thinking about complexity to a type causality setting, where it's assumed you have actual probabilities on co-occurrences.

Limiting Causality by Complexity Class

This post is a mixture of two questions: "interventions" from an agent which is part of the world, and restrictions

The first is actually a problem, and is closely related to the problem of how to extract a single causal model which is executed repeatedly from a universe in which everything only happens once. Pearl's answer, from IIRC Chapter 7 of Causality, which I find 80% satisfying, is about using external knowledge about repeatability to consider a system in isolation. The same principle gets applied whenever a researcher tries to shield an experiment from outside interference.

The second is about limiting allowed interventions. This looks like a special case of normality conditions, which are described in Chapter 3 of Halpern's book. Halpern's treatment of normality conditions actually involves a normality ordering on worlds, though  this can easily be massaged to imply a normality ordering on possible interventions. I don't see any special mileage  here out of making the normality ordering dependent on complexity, as opposed to any other arbitrary normality ordering, though someone may be able to find some interesting interaction between normality and complexity.


Speaking more broadly, this is part of the broader problem that our current definitions of actual causation are extremely model-sensitive, which I find a serious problem. I don't see a mechanistic resolution, but I did find this essay extremely thought provoking, which posits considering interventions in all possible containing models: http://strevens.org/research/expln/MacRules.pdf

Leaky Delegation: You are not a Commodity

Thought I'd share an anecdote that didn't make it into the article: on how doing something yourself can make you a better outsourcer.

About 6 months ago, I went shopping for a logo for one of my projects. It helped greatly that I've spent a lot of time studying visual design myself.

I made a document describing what I wanted, including a mood board of other logos. I showed it to a logo design specialist recommended by a friend. He said "That's the best logo recquisition doc I've seen, and I've seen a lot."

I also showed it to the designer I've been working with on other things (like sprucing up Powerpoint slides). She's not a logo specialist, but quoted half the price.

I had the confidence that I'd be able to give good feedback to my designer even if she was less likely to knock it out of the park on her first try than the specialist. I went with her.

Many rounds of feedback later, I had a design. Showed it to some housemates and my advisor. "Dang, that's a good logo."


You can see it live at www.cubix-framework.com .

Leaky Delegation: You are not a Commodity

Oh, on the contrary: I think this article misses several things that are quite important (or were brushed under a single sentence like "[main principal/agent problems] are communication and risk." Reason: emphasis on things fewer readers were likely to consider.

So the costs you're describing are indeed real and brushed off to corner. I think both of these fall under transaction costs, and #2 also under centralization and overhead. For #2, I think you mean something other than what "externality" means to me (a cost specifically born by a non-party to a transaction) --- maybe second-order cost?


Leaky Delegation: You are not a Commodity

Thanks! This is good.

It's not a physical good, but I had also been thinking that most of the price of renting a venue on the open market is trust (that you won't mess up their space; whether they can give you the keys vs. needing someone to let you in), followed by coordination. Hence, why having a friend let you use their office's conference room on a weekend to do an event might cost $0, while renting such a space might cost $1000.

Leaky Delegation: You are not a Commodity

To clarify: You're not saying the wedding tax is because of insurance costs, as the article is asking about, right?

Help your rivals when they are numerous

I have a number of issues with this post.


First, as others have mentioned, opponents are very much not equal. Further, timing is important: certain trades you should be much more or less likely to take near the end of the game, for example.

Second, I don't think it's valid to look at expected values when all you care about is rank.  Expectation is very much a concept for when you care about absolute amounts.

Third, which perhaps sums everything up: I don't see a valid notion of utility / utility maximization for board games, other than perhaps "probability of winning," which makes this circular ("if you're trying to win, you should make moves that increase your probability of winning"). Utility is meant to put a linear scale on satisfaction with a given state of the world. When discussing what to do in a board game, one usually presumes the objective is to win, and satisfaction derives ultimately from winning. The closest thing you usually see to a "utility" number on an intermediate state is a heuristic, as used in e.g.: chess AIs, where you might give yourself 5 points for having a pawn in a center square. If I'm remembering my undergrad correctly, these heuristics are intended to approximate log-likelihoods of victory, but they certainly lack the soundness required to think about expected utility.


Let's switch out of Catan, and to a game that hopefully people here know but is more directly combative: Diplomacy. Pray tell me how you propose to assign a utility score to putting a navy in the Black Sea.

Load More