Yudkowsky:
You seem to be responding to a bias that studying economics tends to inculcate in people. The bias is to assume that if people follow their preferences then their action cannot be dismissed as stupid or as ultimately having negative effects. Philosophers often accuse economists of this same bias. (A while ago, a philosopher on this blog accused Robin Hanson of not considering deontological reasons for implementing certain health policies. Yudkowsky rebuked Hanson for failing to distinguish between actions beneficial relative to the self-interest of individuals and beneficial relative to humanity during the discussion of Kahneman's article about biases towards Hawkishness).
This suggests a possible general discussion: which biases do various academic disciplines promote? Knowing this would be useful for individuals trying to de-bias (e.g. if I'm an economist, then I should read some ethics from analytic philosophy). I don't know many academic disciplines well, but I'll make some uninformed speculations to show the sort of biases I mean:
Economics: assuming people are more rational than they are (though economists are getting better at this), assuming that markets can be efficient in practice.
Physics: assuming that everything that is not math or physics is (a) soft and easy and (b) not hard or precise enough to give serious epistemological value. Taking the philosophical view that there is nothing real apart from what is studied by physics (e.g. ethics must be BS because in doesn't involve physics, sociology can't be real because it can't be reduced to physics).
Analytical philosophy: Assuming that science will barely move forward in the future. For example, various analytical philosophers have said (following Chomsky) that understanding consciousness scientifically may be beyond are cognitive abilities (the same is also said of understanding ethics scientifically). Yet the same could have been said about 'life' (in the sense that animals and bacteria are alive) or about the origin of life and species. These problems seemed so mysterious as to be beyond the capability of science to explain (e.g. how does self-replication avoid infinite regress?).
Yudkowsky: You seem to be responding to a bias that studying economics tends to inculcate in people. The bias is to assume that if people follow their preferences then their action cannot be dismissed as stupid or as ultimately having negative effects. Philosophers often accuse economists of this same bias. (A while ago, a philosopher on this blog accused Robin Hanson of not considering deontological reasons for implementing certain health policies. Yudkowsky rebuked Hanson for failing to distinguish between actions beneficial relative to the self-interest of individuals and beneficial relative to humanity during the discussion of Kahneman's article about biases towards Hawkishness).
This suggests a possible general discussion: which biases do various academic disciplines promote? Knowing this would be useful for individuals trying to de-bias (e.g. if I'm an economist, then I should read some ethics from analytic philosophy). I don't know many academic disciplines well, but I'll make some uninformed speculations to show the sort of biases I mean:
Economics: assuming people are more rational than they are (though economists are getting better at this), assuming that markets can be efficient in practice.
Physics: assuming that everything that is not math or physics is (a) soft and easy and (b) not hard or precise enough to give serious epistemological value. Taking the philosophical view that there is nothing real apart from what is studied by physics (e.g. ethics must be BS because in doesn't involve physics, sociology can't be real because it can't be reduced to physics).
Analytical philosophy: Assuming that science will barely move forward in the future. For example, various analytical philosophers have said (following Chomsky) that understanding consciousness scientifically may be beyond are cognitive abilities (the same is also said of understanding ethics scientifically). Yet the same could have been said about 'life' (in the sense that animals and bacteria are alive) or about the origin of life and species. These problems seemed so mysterious as to be beyond the capability of science to explain (e.g. how does self-replication avoid infinite regress?).