I am a bit confused and thought I'd rather ask and discuss here before thinking about it for long. As usual I am trying to compartmentalize, structure, make distinctions.
My confusion was triggered thinking about the evaluation function (heuristic to rate the certainty to win/loose) in chess. Clearly what it takes is all there on the board, actually the game is already decided based on that state and assuming both player play to force the best possible outcome.
Why do we need to process data when the information is obviously already in the input? (Yes, I know one can make wordy the distinction between data, information, knowledge, wisdom, understanding.)
Here a few cases, as examples:
Chess: The game state takes few bit to store, yet it takes a lot of processing to decide on a move.
Search: I know what I want but still have to go through a much bigger data set.
Statistical inference: I came up with an hypotheses and now I have to go through tons of data to see if it is consistent.
Automation: I am trying to control something and must derive actions from measurements.
I suppose any compute is just a replacement for a huge LUT, proving that the information is already in the input, but then searching that LUT would still take compute.
There are transformations that just change the perspective, but do not discard or compress.
There is compression were I either only care for decisive aspects (filter / lossy) or there is redundancy in the input (lossless).
In the end it is always about deriving efficient actions to reach goals. For that, I suppose, we can make a one fits all scheme: Make and maintain model. Measure current state. Filter measurement. Compute possible outcomes. Evaluate possible outcomes impact and likelihood. Decide on / search for best action.
But what I find so confusing is the contrast of starting out with little information or a lot but that does not explain alone how much there is to compute. Why can I have little information but still have to search a huge state space, why can't I go straight to the conclusion/action?
Obviously I am thinking about it wrong, there might not be an antagonism but a confusion of stages.
Thank you, I should have thought of it in that (Time complexity) context. Time complexity is not just about how long it takes but also about the nature of the problem. Chess is neither P nor NP, but the question of complexity is certainly related.
Maybe my question is: Why can there be a Heuristic that does fairly well and is nowhere near exponential? Even a count of the pieces left on the board usually says something that only a full search can prove.