This is a special post for quick takes by Eli Tyre. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I'm mostly going to use this to crosspost links to my blog for less polished thoughts, Musings and Rough Drafts.
Doing actual mini-RCTs can be pretty simple. You only need 3 things:
1. A spreadsheet
2. A digital coin for randomization
3. A way to measure the variable that you care about
I think one of practically powerful "techniques" of rationality is doing simple empirical experiments like this. You want to get something? You don't know how to get it? Try out some ideas and check which ones work!
There are other applications of empiricism that are not as formal, and sometimes faster. Those are also awesome. But at the very least, I've found that doing mini-RCTs is pretty enlightening.
On the object level, you can learn what actually works for hitting your goals.
On the process level, this trains some good epistemic norms and priors.
For one thing, I now have a much stronger intuition for the likelihood that an impressive effect is just noise. And getting into the habit of doing quantified hypothesis testing, such that you can cleanly falsify your hypotheses, teaches you to hold hypotheses lightly while inclining you to generate hypotheses in the first place.
Theorizing methods can enhance and accelerate this process, but if you have a quantified empirical feedback loop, your theorizing will be grounded. Science is hard, and most of our guesses are wrong. But that's fine, so long as we actually check.
I'm mostly going to use this to crosspost links to my blog for less polished thoughts, Musings and Rough Drafts.