This problem seems to me to have the flavor of Moloch and/or inadequate equilibria. Your criticisms have two parts, the pre-edit part based on your personal experience, in which you state why the personal actions they recommend are actually not possible because of the inadequate equilibria (i.e. because of academic incentives), and the criticism of the author's proposed non-personal actions, which you say is just based on intuition.
I think the author would be unsurprised that the personal actions are not reasonable. They have already said this problem requires government intervention, basically to resolve the incentive problem. But maybe at the margin you can take some of the actions that the author refers to in the personal actions. If a paper is on the cusp of "needing to be cited" but you think it won't replicate, take that into account! Or if reviewing a paper, at least take into account the probability of replication in your decision.
I think you are maybe reading the author's claim to "stop assuming good faith" too literally. In the subsequent sentence they are basically refining that to the idea that most people are acting in good faith, but are not competent enough for good faith to be a useful assumption, which seems reasonable to me.
Typo: "Systemic reviews" should read "systematic reviews".
The article about this on Strengtheory has links to sources (not as footnotes, in the text). May be useful to check out.
When it comes to problems that are primarily related to motivation, the cost-benefit is so far weighted that the cost of implementing the plan probably doesn't seem relevant to consider, but this is a good point.
I like the idea of using Murphyjitsu for modeling shorter iterations, that's probably generally applicable.
That seems mostly about the emotional content of a particular plan, while I see Murphyjitsu as a tool for avoiding the planning fallacy, forcing yourself to fully think through the implications of a plan, or getting more realistic predictions from System 1. I haven't viewed it much as an emotional tool, but maybe other people do find it useful for that.
Whew, glad I didn't invest more time in this. Seems there is lurking complexity everywhere.
At this price point this seems potentially doable. Some ideas in the order I'd try them:
This is a pretty fun format. Actually, I really like this gomboc idea and briefly considered doing a Kickstarter on it after reading your post. But then I realized that Kickstarter would only really make sense if everyone were willing to pay $800. The market is so niche, that it would have to be a passion project to be worth the hassle I think.
I admit there might be reasons to invest in meditation practice that are not based on scientifically proven benefits (e.g., curiosity, sense of novelty, sense of belonging to a community). At the same time, I hope that most LW readers attach very little weight to those non-evidence-based reasons to meditate, just like I do.
I suppose I should admit the main reason I started meditating a long time ago was curiosity. I read Mastering the Core Teachings of the Buddha (reviewed on SSC here) and thought "well, this person sounds like they are explaining mental states that seem pretty unbelievable to me, I wonder if this is all BS." I was/am mentally healthy and emotionally stable more than the average person. I don't meditate that consistently anymore, only when things are more stressful than usual. Having it in the toolbox, like fitness, is enough for me. I did enough practice to know that what MCTB is pointing at is a real phenomenon, but that's it. I actually think that viewing it as a hobby is the healthiest way to approach the kind of serious practice needed for enlightenment.
Let's start with the easy to verify claims that I generalise as...
In my experience, these claims are false. I occasionally tried to use mindfulness to help me with dieting or exercising, since those are also things I do, and it never helped in a way I could discern.
Do you have some sources to back this up? I've heard many declared reasons why people begin their meditation practice, and it was quite a diverse set, none seemed dominant.
Thank you for challenging me on this, that was based only on personal observation, which as I admit above doesn't even square with my own experience! This survey has concrete data on why people meditate in Fig. 1. The top reason is "General wellness and general disease prevention." None of them are specifically happiness related, so maybe that's an overly specific claim.
I don't buy this at all. If the only observable benefit of me meditating is that I used to self-report average well-being of 5.17 out of 10, and now I self-report 7.39 on average
Based on my mental model of meditation, you probably would be dissatisfied with the results. In section IV of the post above, Scott Alexander summarizes thus:
Ingram dedicates himself hard to debunking a lot of the things people would use to fill the gap. Pages 261-328 discuss the various claims Buddhist schools have made about enlightenment, mostly to deny them all. He has nothing but contempt for the obviously silly ones, like how enlightened people can fly around and zap you with their third eyes. But he’s equally dismissive of things that sort of seem like the basics. He denies claims about how enlightened people can’t get angry, or effortlessly resist temptation, or feel universal unconditional love, or things like that. Some of this he supports with stories of enlightened leaders behaving badly; other times he cites himself as an enlightened person who frequently experiences anger, pain, and the like. Once he’s stripped everything else away, he says the only thing one can say about enlightenment is that it grants a powerful true experience of the non-dual nature of the world. [9eb1: I've excluded the possible counterargument here for brevity]
There are external benefits I think meditation has given me that feel like they are real, but the effect size is too small for studies to realistically find them. I can fall asleep reliably by using meditation as a tool. I can tactically break my own rumination thought cycles by meditating as a tool (or I can workout, but sometimes you've already worked out that day). I definitely feel like I am harder to surprise (lack of "jump"), but that's not a particularly practical superpower.
I've meditated for >300 hours (maybe 4 or 5). I don't regret my hours. It is a hobby, it satisfies my curiosity, it makes me happy when I need it. Lack of personal transformation is fine.
To be clear, my values with regard to self-rated wellness are different from yours. I am glad to improve my self-rated wellness even if it has no measurable outward impact on my behavior. My happiness is super important to me. If I move the needle on that, that's great even if I'm still an asshole. I have no interest in being a miserable saint.
There are several characteristics of nutrition and eating that make scientific scrutiny very difficult, and those characteristics are not shared with meditation
Those differences are subsumed in the "high short-term costs" side of my statement. The exact costs are different, that's all. You can tell people not to do all the things you mentioned during a diet study, but they won't follow your instruction.
I think it is right to be skeptical of the science around meditation. Meditation perfectly fits into the Bermuda triangle of phenomenon for which our current scientific institutions and practices are not well-prepared to study.
It shares with psychological studies the challenge that the thing under investigation is the internal mental state of the subject. When there are studies with objective endpoints, usually the objective endpoint isn't the thing we want to get out of it, it's just a more reliable metric so we know the subjects aren't fooling themselves. As Science-based Medicine says:
But the more concrete and physiological the outcome, the smaller the placebo effect. Survival from serious forms of cancer, for example, has no demonstrable placebo effect. There is a “clinical trial effect,” as described above – being a subject in a trial tends to improve care and compliance, but no placebo effect beyond that. There is no compelling evidence that mood or thought alone can help fight off cancer or any similar disease.
In the case of meditation, people usually begin the practice to have mental well-being or greater happiness, which is among the outcomes least amenable to reliable objective observation. If it happens to also do something that could be reliably measured with a medical instrument, that would be a bit outside the point.
Meditation shares with nutritional science (also a wrecked landscape of low-quality studies that fail to answer our real questions) that performing the study relies on the subjects to reliably do something with a huge, short-term cost and an uncertain, long-term benefit, which humans are bad at.
High-quality studies on nutritional interventions rarely answer the questions that normal fitness-minded folks want answered, because we want the answer to the question "assuming I perfectly adhere to diet X, what results would I obtain." Studies can only measure "assuming we take a random sampling of people with varying levels of conscientiousness and investment in their diet, and tell them to do X, what happens" which is too big a difference to be useful.
Similarly with meditation, what meditators want to know is "is it worth my time meditating if I do it approximately perfectly" not "is it worth someone 'intervening' to tell me about meditation taking into account the possibility that I'm too lazy to really follow through with it." The second has more clinical relevance, but less personal relevance for the kind of people on Less Wrong.
All of that is a long precursor to saying "Is there any scientific evidence for benefits of meditation?" and "Are there good reasons for a typical reader of LessWrong to invest their time and effort into meditation practice?" are subtly different questions, so it would be wrong to literally equate them. The second is the answer we really care about, the first is one input which would, if available, fully resolve the question instead of leaving is in a state of uncertainty. You're entitled to arguments, but not (that particular) proof.
There is objective evidence that meditation does something real (EEG studies of Tibetan monks, for example), but the evidence it does something both real and valuable is probably not up to that standard.
Another, smaller, point I'd like to make is that this post is attempting to perform its own meta-analysis, but with a higher quality bar than academic meta analyses. I don't think crowdsourcing the best studies of meditation is likely to work this way. If you are interested in running a project to identify the top studies of meditation, I think you would need to identify all the relevant studies, get individuals who are interested in your project to review them, then collate the results. Just asking "the crowd" for the best studies they happen to have on hand I think is likely to fail regardless of what the evidence is.
They have to do it to some extent, otherwise replicability would be literally uncorrelated with publishability, which probably isn't the case. But because of the outcomes, we can see that people aren't doing it enough at the margin, so encouraging people to move as far in that direction as they can seems like a useful reminder.
There are two models here, one is that everyone is a homo economicus when citing papers, so no amount of persuasion is going to adjust people's citations. They are already making the optimal tradeoff based on their utility function of their personal interests vs. society's interests. The other is that people are subject to biases and blind spots, or just haven't even really considered whether they have the OPTION of not citing something that is questionable, in which case reminding them is a useful affordance.
I'm trying to be charitable to the author here, to recover useful advice. They didn't say things in the way I'm saying them. But they may have been pointing in a useful direction, and I'm trying to steelman that.
Even upon careful rereading of that sentence, I disagree. But to parse this out based on this little sentence is too pointless for me. Like I said, I'm trying to focus on finding useful substance, not nitpicking the author, or you!