[ Question ]

The Right Way of Formulating a Problem?

by Virgil Kurkjian 2mo2nd Jul 20194 comments

16


David Chapman Writes,

Finding a good formulation for a problem is often most of the work of solving it.

I agree with this intuitively, and I feel like I have seen this principle at work in my own work and in the problems I have tried to solve. However, when I try to convince others of this idea, I struggle to find examples that they can connect with or that they find compelling.

I suspect that programmers find this idea appealing because we routinely work with formal systems, and all of us know the experience of making a minor change in perspective and seeing an impossible problem turn into an easy one. So I'm most interested in examples that have nothing to do with code, examples that a lay audience would be able to grasp.

I would be particularly interested in examples from the history of science or medicine, if anyone can think of some. Scott and Scurvy is the only example I currently know of, and while interesting, does not seem like a perfect fit.

Much appreciated!

New Answer
Ask Related Question
New Comment

1 Answers

Here's a programming example which I expect non-programmers will understand. Everyday programming involves a lot of taking data from one place in one format, and moving it to another place in another format. A company I worked for had to do even more of this than usual, and also wanted to track all those data flows and transformations. So I sat down and had a long think about how to make it easier to transform data from one format to another.

Turns out, this sort of problem can be expressed very neatly as high-school algebra with json-like data structures. For instance, you have some data like [{'name':'john',...},{'name':'joe',...},...] and you want to extract a list of all the names. As an algebra problem, that means finding a list of solutions to [{'name': X}] = data. (Of course there's simpler ways of doing it for this simple example, but for more complicated examples with tens or even hundreds of variables, the algebra picture scales up much better.)

Problem formulation is even more important in data analysis and/or machine learning problems. At one company I worked for, our product boiled down to recommendation. We had very fat tails of specific user tastes and corresponding items, so clustering-based approaches (i.e. find similar users, recommend things similar to things they like) gave pretty mediocre recommendations - too many users/items just weren't that similar to any major clusters, and we didn't have enough data to map out tiny clusters. Formulating the problem as pure content-based recommendation - i.e. recommending things for one user without using any information whatsoever about what "similar" users were interested in - turned out to work far better.

Anyway, that's enough from my life. Some historical examples:

... etc. Practically any topic in applied math began with somebody finding a neat new formulation.