[ Question ]

What questions should we ask ourselves when trying to improve something?

by weathersystems3 min read6th May 20219 comments

3

World Optimization
Personal Blog

I'm trying to come up with a template for questions of the form "How should we improve X?"  I assume most such questions will involve answering many subquestions. Sometimes the answers to those subquestions will be implied in the context, or the questions may not all be relevant. But my guess is that the larger and more complicated the system is that would be improved, the more you'd want to explicitly answer these subquestions. 

There are many ways you may be able to help me. This question is a bit meta, in the sense that I'm trying to improve something (the template). Here are the questions from the template that seem relevant here. Feel free to contribute to any of these:

 

What is the status quo?

Here's the template so far:

  • What is the status quo?
    • What are the strengths of the status quo (that we want to try and keep)?
    • What are the problems?
      • Given these problems, why are people still tolerating the status quo (if they are)?
  • What are the goals we are aiming for?
    • Possibly ordered by easiest to most difficult to achieve.
  • What are the goals we shouldn't aim for?
    • Often times setting up non-goals is important to guard against scope creep.
  • What resources might help in making progress in X?
    • What information might be relevant?
      • Is there scientific research relevant to the question?
      • Are there relevant arguments presented in articles or online discussions?
    • Who may be able/willing to help?
  • What are the possible improvements to X?
    • What actions are currently in progress that attempt to improve X?
    • What are the strengths and weaknesses of each option?
      • How likely is it that the proposed improvement would make things worse?
    • When will each proposal be ready to be acted on?
      • Are there stakeholders that should sign off on the proposal?
    • What has been tried already?
      • Why did it succeed/fail?
    • What outcomes are/should we measure in determining the success of this action?

 

What are the goals we are aiming for?

I'd like to have a template that could be used to start new question pages that have the form "How should we improve X?"

The template should be applicable across a wide range of progress type questions.

The template should feel useful to people asking these sorts of questions.

 

What are the goals we shouldn't aim for?

We shouldn't aim to cover every single possible question that could be relevant to the top level question. That would be too overwhelming.

 

What resources might help in making progress?

What information might be relevant?

Process Improvement Theory and Application (maybe this is relevant?)

Who may be able/willing to help?

reddit? what subreddits might be able to help?

 

What are the possible improvements to the template?

 

What outcomes are/should we measure in determining success?

I guess anyone who wants to test the thing can try to use the template and give feedback on if it was helpful or not.

 

What are the possible difficulties in making improvements in this area?

It will be a tough balance to strike to have a template that's sufficiently general such that it applies to many situations, but at the same time has enough detail to have some marginal value over just asking the question without the template.

 

Are there any analogous systems that we might look to for inspiration?

RFC's seem to have a similar structure to "how do we improve X" type questions.

Process Improvement theory may be close enough to be relevant.

[from nim]: "Science of decision-making may be interesting, as it tends to be predicated on the assumption that decisions need to pick or create the "best" option, aka an improvement."

3

New Answer
Ask Related Question
New Comment

2 Answers

There's a trade-off to be weary of here in trying to improve the list only by adding. Making it complete might also make it too long to actually get much use.

Can you optimize the list to keep the most valuable bits while also keeping it nice and short to increase the chance someone will go through the effort of answering the questions?

I noticed in your own use of the template, you only included a few of the sub-questions.

I was thinking that the template would be something where you could just keep the sections that seem relevant and delete the rest. 

But I guess even that would start to get annoying if the thing was super long. That's a good consideration to keep in mind.

I added in a few more of the questions from the template that seem relevant. Including the one about possible difficulties. I think what's there cover's your trade-off.

A suggested improvement to the template: When examining the status quo, also ask "for what related problems does the status quo have a built-in solution?".

You have spelled "stakeholders" as "steak-holders", which is charming but may reduce credibility in some circumstances.

The template might benefit from a section asking what preconceptions or stereotypes surround the topic. When I think back to times I've tried to improve things, I recall some being stymied by a group of people having a social memory of a time that something related was tried and went horribly wrong, or a time that some other group tried a similar thing and it backfired, or other expectations of negative outcomes.

The other question to consider adding, possibly adjacent to "what has been tried already", is along the lines of "why am I unwilling to tolerate this status quo when everyone else experiencing it does tolerate it?". I don't think this is the best phrasing, but comparing one's experience with something intolerable to the experiences of those who find it tolerable can often reveal techniques for making the experience tolerable, which shifts the value proposition of eliminating the experience.

What resources might help in improving this template?

Science of decision-making may be interesting, as it tends to be predicated on the assumption that decisions need to pick or create the "best" option, aka an improvement.

You have spelled "stakeholders" as "steak-holders", which is charming but may reduce credibility in some circumstances.

Heh. Funny mistake. Thanks.

A suggested improvement to the template: When examining the status quo, also ask "for what related problems does the status quo have a built-in solution?".

I want to make sure I understand your point here. Is the idea that sometimes we see that a system isn't solving some problem well enough, and so try to fix it. But we don't take into account the fact that the system isn't just trying to solve that problem, but ... (read more)

2karlkeefer3moNot OP, but I read their comment about related problems as something more like this: The system in question likely already has feedback or correction mechanisms that respond to other potential problems - asking about those mechanisms might reveal strengths of the system that can be easily adapted for your purposes. I'm not sure how easy it will be to find these, though, as the best-functioning ones might be invisible if they actually eliminate the other problems completely. That might not be their intent, but I think it's also a useful consideration so even if my interpretation isn't matched I hope this comment is still useful :)
1nim3moYikes, I see why -- I worded the concept quite poorly. The example I was trying to describe is in software engineering, where you have an ancient crufty mess that you're trying to rewrite in some snazzy new language. You think you can rewrite it and make it super simple, and so you write the new thing the simple way that "should work", but when you run the old code's tests against it (or when you put it to use in the real world...) you discover that the reason the old code was such a mess was partly that it had a bunch of logic to handle various edge cases that the application had hit in the past. An alternative phrasing might be: "Where are the gaps between how I think the status quo 'should' work, and how it actually does?". Often, established systems are silently compensating for all kinds of problems that happen infrequently enough for any one person to forget that the problem exists when trying to replace the system.

I added "Given these problems, why are people still tolerating the status quo (if they are)?" to the template. Does that capture your idea well enough?

2nim3moI think that's a good snapshot of the concept I'm trying to get at. It asks what benefits the status quo may silently be providing, which a competitor would have to match or exceed to gain acceptance.