I'd like to suggest an alternative learning model view, one less concerned with getting immediately an intimate and "true" understanding of the reality, and more focused on with progress through stages.
The first stage is awareness. When we are exposed to light theories, particle theory, wave theory, and later quantum theory, we clearly don't know what a light particle is, so we attach verbal labels to ideas. This is not perfect, but allows us to learn the concepts and start thinking about it. Part of this process is "guessing the teacher password". When we guess right routinely, that is we associate the right visual stimuli with the right verbal label, we are ready to go to the next stage.
The next stage is making some calculus and some predictions. Here we are deepening our understanding and our model of reality by adding some equations. Again, a process of learning at the end of which we clearly do not have a perfect or true understanding, but its better than the first stage as it allows us to do more.
The following stages are about refining our models and making them more accurate, better predictor and more useful for surviving (or getting a Nobel prize).
In the end, our thinking will always be constrained by our models we will probably never have a perfectly true understanding of light. But going throuhg school put us on the path on building progressively more elaborate and/or useful models to operate in the world.
"And remember that General Relativity was correct, from all that vast space of possibilities."
Well... actually, one thing I feel pretty confident about is that General relativity is wrong. It is an approximation which works well within a large domain, but at distances and energies where quantum theory is a better description, it does not work. Hence the search for the quantum theory of gravity which has been going on for some time.
In the same way that Newtonian theory is an approximation of Einstein theory for non relativistic speeds, Einstein theory is probably an approximation of this yet to be discovered quantum theory of gravity, which should help us understand how black holes work.
Am I the only one unconfortable with this example ?
In all games of chance, the issue is not about winning or losing once. It's about the probability of winning and the expected value of betting on the long term.
So if you have 1/132 chance of winning, but you win 10 millions times your bet, you should be willing to bet as much as possible and the probability you would be a winner is better than 50% (in money won, not times played and won)
Same with poker, an expert player is never guaranteed to win, expert players are maybe 60/40 favorites to win over bad players, and after 100 hands, they are huge favorites to end up with more money.
Now, about quantifying the number of bits of information to prove a scientific theory, you would need to know the number of possible theories (one correct theory and all the others wrong). Since the number of theories which are incorrect can be made infinite, quantifying the number of bits seems to me an unsatisfying approach to quantify how much a theory is likely to be true.
Thanks Eliezer,
I am surprised that you only have three motivations for seeking out truth in your conclusion. Moral duty, pragmatism and curiosity. Even though you talk about manipulating the world while talking about curiosity.
I would separate curiosity, where the benefit is enjoyment at understanding, and power seeking, which allows shaping the world more efficiently.
Certainly in the scientists I know, those motivations are often mixed. The search for exotic particles in physics is closer to curiosity and the "pleasure of finding things out".
The quest of seeking the truth in applied physics to build a nuclear bomb has more to do about power seeking.
Two very different motivations, no ?
"What distinguishes a semantic stopsign is failure to consider the obvious next question."
Why does one fail to consider the obvious next question ? I believe it is often due to fear.
Fear of the unknown is allayed by tricking the mind into believing a societal explanation, "God", which is less scary than "I don't know".
Fear of looking like an idiot is allayed by tricking the mind into believing what "everyone knows". It reminds me of a quote attributed to George Leonard:
“Man”, he said, “you are a learner. Tell me. How can I be a learner?”
“It’s simple. To be a learner, you’ve got to be willing to be a fool.”