Subspace optima

by chrisvm 1 min read15th May 20206 comments

42


The term "global optimum" and "local optimum" have come from mathematical terminology and entered daily language. They are useful ways of thinking in every day life. Another useful concept, which I don't hear people talk about much is "subspace optimum": A point maximizes a function not in the whole space, but in a subspace. You have to move along a different dimension than those of the subspace in order to improve. A subspace optimum doesn't have to be a local optimum either, because even a small change along the new dimension might yield improvements. If you're in a subspace optimum, this requires a different attitude to get to a global optimum, than if you're in a local optimum, which makes me think it's good for the term to be part of every day language.

  • When you're in a local optimum, you have to do something quite different from what you're doing to improve.
  • When you're in a subspace optimum, you have to notice dimensions along which you could be doing things differently that you didn't even notice before, but small changes along those new dimensions might already help. You're applying constraints to yourself that you could let go.

Regarding how it looks subjectively:

  • The phrase: "am I in a local optimum?" generates curiosity about whether you maybe should undertake a quite different plan from the one you're taking now. (Should I do a different project, rather than make local changes to the project I'm taking?)
  • The phrase: "am I in a subspace optimum?" generates curiosity about whether you maybe are not noticing (possibly small) changes you could be making across dimensions you haven't been considering. (Should I optimize/adjust the way I'm doing my project across different dimensions/variables than the ones I've been optimizing over so far?)

My impression is that somewhat often when people informally use the term local optimum, they are in fact talking about a subspace optimum.

Bonus for the theoretically inclined: A local subspace optimum is one where you can improve by temporarily doing things differently along dimension X, moving around in a bigger space, while eventually ending up on a different, better, point in the same subspace.

42