One of the heuristics you see in the business world that attempt to get at this is the "5 Whys?" It's very easy to look at some graph- stock value, sales, whatever- and create a just-so answer for why something is the way it is. It's a lot harder but more useful to go ahead and interrogate the just-so answer again.
Of course, the hardness of doing a "5 Whys?" exercise is also the reason that nobody does it unless they're getting paid to or they've joined an online cult of critical thinking.
For obvious reasons, average westerners have a comparative advantage in money-making relative to the average human. Thus, westerners (usually) shouldn't directly volunteer their time to altruistic organizations which work in e.g. the third world; They should perform the two-step resource conversion of My Time -> Money -> Someone Else's Time.
Some people in the third world are scammers. If you send your money without spending your time to get familiar with the situation, there is a risk you send to one of those. The more people send money without making research, the more profitable scamming becomes.
Now that effective altruism is a thing, this is solved by outsourcing your research to GiveWell. But before that, volunteering your time was one way to find out. And the optimal strategy would be to volunteer shortly, then switch to making money and donating. Which is kinda what people did, because volunteering is one of those things people usually do during university, when they don't make lot of money anyway.
I've been thinking about the differences between the ways in which people typically pursue wealth and the ways in which people typically pursue their altruistic goals. Charity is significantly signalling, and so on... But, there's something else I noticed.
Whenever you want wealth, you pursue a high paying career, open a business, and so on. Whenever you (a hypothetical average person unfamiliar with the rationalist or effective altruism community) want to accomplish some altruistic goal, you're going to consider personally volunteering yourself to perform the associated cheap labor. If you decide to donate instead, you probably see it as the less impactful option. It would be better (you would think) to go to third world villages yourself than to donate to charity.
The underlying reason behind this mistake is the notion of topicality- proximity in an unconsciously constructed web of connections. Traditionally, different paths to power (business, government, personal charitable work, etc.) are associated with different usages. Volunteering is for helping starving children, private business is for making money, government work is for changing what the government does. This association is often valid- different types of power are better at accomplishing different goals.
It's less valid when those types of power are highly interchangeable. Imagine money and political power have a standardized rate of exchange in the vein of one US dollar per DemocracyCoin. It's either going to be better to acquire money to buy political power, or it's going to be better to acquire political power to buy money- manipulating the government isn't easier if you're a politician, and buying gold jewelry isn't easier if you're a business owner. One profession or the other is better at both.
In the case of standard employment (the trade of your man-hours for the money of others) and charity (the trade of your money for the man-hours of others), there are clear exchange rates. For obvious reasons, average westerners have a comparative advantage in money-making relative to the average human. Thus, westerners (usually) shouldn't directly volunteer their time to altruistic organizations which work in e.g. the third world; They should perform the two-step resource conversion of My Time -> Money -> Someone Else's Time.
More generally: In a big and complicated world where so many resources are directly convertible and generally useful, it's highly unlikely that the best human-level strategies for accomplishing medium to long term goals are composed out of steps which are in any way "topical"- superficially related in the way that starting a charity that plants trees is related to environmental concerns but (for example) being a computer science professor at a prestigious university isn't.
The mental mechanisms behind the unconscious associations we draw, and thus the borders of topicality, pale in comparison to our deliberate planning ability. Making your plans out of a handful of those unconscious associations which you've haphazardly cobbled together is a bad start- there is only so much genius in a single step.
Much of the intellectual progress of slightly younger me was in replacing bad low level ideas with better ones- changes analogous to deciding to work and donate to a charity which plants trees instead of planting trees myself. But, both are bad ways to save the environment. The strategy behind their generation (stringing together some select ideas that floated to the surface of my wandering mind) was bad, so all such strategies are cursed by methodological badness. This effect doesn't go away if you're really smart- these strategies are always going to be bad relative to what you can produce with better methods.
There's value in slowly refining low level ideas (working and donating > volunteering) by "just thinking" about the topic, but only so much. There's much more value in developing an idea of why you were previously mistaken, of which other legacy opinions are affected by the same methodological error and will need to be reevaluated, and more importantly in developing and using methods of reasoning which don't let individual cached thoughts represent single points of failure.
You can't develop a heavier than air flying machine by "just thinking" about flight. Human brains are just not advanced enough to successfully perform those types of tasks without building huge sequences of successive layers. There's, as a zeroth order approximation, hundreds of layers of theories (the borders are fuzzy, of course) which are necessary to build a primitive heavier than air flying machine, starting from the peak science of the year 2000 BCE.
Thinking about flight and taking what your surface thoughts give you- birds, feathers, flapping wings- will not cut it. As a start, you need something like classical mechanics, which will require a bunch of things including a complicated theory of mathematics and something like the scientific method, which will require blah blah blah.
Similar concepts follow if you want to do good altruistic work. You can't just make up an answer in the same way you can't just make up a design for a heavier than air flying machine. As a start, you need a theory of what you value and a theory of what the world looks like, which will in turn require such and so on.
You realize there are on the order of 10^20 stars out there whose future usage you're partially determining, right? Well, that's a bit of an underestimate... That's the sort of thing which you need to include within your altruistic decision making. You can't expect to have a good strategy if you didn't think about that!
Philosophy, Politics, Altruism, and so on- some questions deserve correct answers. So, think deliberately. Divide questions into trees of subquestions whose answers can be repeatedly recombined to help answer the original question. You can't just have an answer, directly delivered from the surface of your idle mind- an answer built from thoughts which are not themselves the children of a highly relevant deeper theoretical understanding. It's going to be garbage if the question is complicated. You need to build a good answer.
Is this too obvious? For the general public, absolutely not... Somehow? For the reader of this post, I hope the extra clarity provided by (at the least) reading this idea instead of thinking it is useful.