I know I'm really late with this, but what do you consider as "studying science"? Making a career of it? Does being an engineer count (I guess it does)? Or is getting (an amount of knowledge equivalent to) a B.Sc. enough too? Maybe even less than that, learning cool nuggets of science as a hobby? I think this should be better defined.
If it's just a career that counts, I'm afraid that the main inhibitor is not interest, but fear for career prospects. Most often when I head people's reasons not to pursue a career in science, it's because they don't think they'll make a good living out of it, or because it's hard and they don't think they'll make it. If it's the hobbyist population you're worried about, I think it's pretty decent, after factoring in access to prerequisite knowledge, free time, and upbringings. Though there is a LOT of room for improvement on that front.
Those who actually don't find science interesting seem to think that way mostly because of bad teacher experiences or the social stigma of "nerds", as far as I've seen.
Then "Gomboc righting itself when on a flat surface" will have an inherent 100% probability. This doesn't refute the example.
Three things bother me here, and they're all about which questions are being asked.
The "tree falling in a forest" questions isn't, as far as I've encountered it outside of this blog, about the definition of sound. Rather, it's about whether or not reality behaves the same when you do not observe it, an issue that you casually dismissed, without any proof, evidence, or even argument. There are ways to settle this dispute partially, though they are not entirely empirical due to the nature of the conundrum.
Ignoring the question of free will, ill defined as it may be, is merely -pretending to be wise-. You're basically saying you now know not to ask these questions, without explaining why (at least here). If there are any convincing arguments that settle a well-defined notion of free will, I welcome them.
Last but not least, I'm bothered by the choice of a question to settle all arguments - just write the mental processes that lead to the argument? Why stop there? Why not map the specific clusters of neurons and synapses activating the argument and reinforced by it? Having written down this stack of processes, can you perform neurosurgery that will stop this pattern of thinking (but not unrelated ones)? In Science, there may be such a thing as "being done", but this isn't it. Not by a longshot.
There's a clarification to be made here, in the bottom line - you were right to say that you shouldn't be expected to believe that the big, elaborate argument violates known laws of physics if no specific step had been shown to do it, but this doesn't mean that no such step exists. It may be that the arguer (and anyone else, for that matter) doesn't understand a subtlety that allows the mechanism to coexist with the laws of Nature. This has happened with the proposition of the ERP experiment, when it was initially thought to violate causality, but it was later understood that there's a distinction between causality and locality. This is also the case in many breakthroughs in engineering. The chance of this, while small, is by no means negligible, since it has happened numerous times in the past.
All that said, in the bottom line the burden of proof still lies with the claimer of a breakthrough or violation of physics.
No. 6 - I go again to logic and formal math, where you can never define any term by extensions because sensory perceptions aren't reliable enough to give the needed certainty of Truths. Then you will have to start from some undefined elementary terms and work up from there. Other than this, though, this rule of thumb seems quite trustworthy.
No. 29 - that's just inaccurate. As you said, there are more and less typical examples of a cluster. Hinduism is a typical example, so we stop there. But if a case is a borderline member of a cluster, you will need to run it by the definition to know for sure. And sometimes this will be more reliable or feasible than checking the desired query directly. Whether atheism is a religion will then depend on the definition of religion, which in turn SHOULD depend on the purpose of the categorization.
No.30 - maybe I have a use for "animals that look like fish". "Belonging together" is not such a trivial matter, And there is sometimes serious merit for reclustering.
But it's still the listmaker's responsibility to show that the list has value.
I feel the need to address the python vs. modern art thing too - if you just compare the extensional list of art against the intensional definition, you'll see that modern arts pass as arts (at least sometimes) while python definitely doesn't. Modern arts involve some work, are intended to inspire aesthetic emotions, and often do in some people experiencing them. Python, while being an elegant tool, was not (probably) designed with the primary intention of producing emotions, but rather with the intention of being a convenient tool to code.
Also, there is a legitimate quest of finding the "right definition" of a word, as in what concept it represents. Even if there is no class corresponding to it in reality (e.g. God) the existence of the word means some people treat it as a meaningful concept. If enough people use the same word with enough gravitas, and you want to talk to them about it, you will need to understand what their common ground of the idea is. Even if, as with free will, you arrive at the conclusion that there is no common ground to speak of. Not as interesting as carving reality, perhaps, but if you are somewhat interested in what other humans think, it does have merit.
Well, I think that if you are to be true to the message here, you should go even if the students and professors themselves are not above the norm, since the culture of addressing the original purpose directly would have merit in its own right. Unless you believe this expenditure of time isn't worth the while without the bundled social benefits of having a degree.
As for the PhD level, I think that after that the teaching part is nearly gone, and the service the institution can provide is mostly providing a productive environment and tools to conduct research.
On a different note, calling a ball a spheroid isn't really tabooing it, it's just a synonym.
While the general argument is valid, I'm not sure how these accusations of socially-derived rules making up traditional rationality. There were many mathematicians and scientists before Bayes was born, and they derived their beliefs from logic and evidence, not social norms. Take Galileo as an extreme and famous example. Is there any evidence behind these unflattering descriptions of traditional rationalists?
This "if" embodies the decrease of risk from being part of a crowd. In a protest of 5000, 20 may be pulled in, but the leader is much more likely to be one of them than any one person in the crowd.
I agree with the benefits of narrowness, but let's not forget there is a (big) drawback here: science and math are, in their core, built around generalizations. If you only ever study the single apple, or any number of apples individually, and not take the step of generalizing to all apples, or maybe all apples in a given farm, at least, you have zero predictive power. The same goes for Rationality, by the way. What good is talking about biases and Bayesianism, If I can only apply it to Frank from down the street?
I'm arrogantly confident you agree with me on this to some level, Eliezer, and just were not careful with your phrasing. But I think this is more than semantic nitpicking - there is a real, hard trade-off at play here between sticking to concrete, specific examples on which we can have all the knowledge we want, and applying ideas to as many problems as possible, to gain more predictive power and understanding of the Laws of Reality. I think a more careful formulation is to say "do not generalize irresponsibly". Don't abandon the specific examples, as they anchor you down to reality and details, but do try to find patterns and commonalities where they appear - and pinpoint them in precise, well defined, some-result-subspaces-excluding manners.