Note, if you donate blood, they will test your hemoglobin levels before you donate.
(I donate for both altruistic and selfish reasons.)
Thanks for you work on this! I eat vegan since ~9 years now and also am probably not investing as much time into checking my health as would be optimal, but at least my recent shotgun blood test didn't turn up any issues except being low on HDL cholesterol and I don't have the impression that I'm less healthy than non-vegan peers (but certainly always could feel fitter and more energized than I normally am). Happy to share the test results privately if that'd be useful for you, have one from ~7 years ago and one from this year.
You say you didn't care about age and sex, but I'm curious about the distribution in your participants. Menstruation is very relevant to iron deficiencies.
Fyi - this series of posts caused me to get a blood test for nutritional deficiencies, learn that I have insufficient vitamin D and folic acid, and take supplements on a bunch of days that I otherwise would not have (though less often than I should given knowledge of a deficiency). Thanks!
The happiest people were the ones who got their results over email, which is a HIPAA violation the company claims to have done by accident
this might not be true? it seems ok if patient requests
Thanks for doing this! These results may affect my supplementation strategy.
My recent blood tests (unrelated to this blog post) -- if you have any thoughts on them let me know, I'd be curious what your threshold for low-but-not-clinical is.
(I have other results I can send you privately if you want, from comp metabolic panel + cbc + lipid panel + D + B12; but didn't think to ask for iron. Is it worth going back to ask for this? or might iron be under a name I don't recognize?)
I'm vegan and have been solidly for > 1 year. Generally feel good, no particular fatigue except sleepiness after I eat carbs for lunch. I supplement B12, omega-3 EPA+DHA algae oil, creatine and occasional D3 gummies.
I can't give that kind of individual advice. For ferritin (the best proxy for cellular iron) I gave some guesses here but those are more about returns to treating very low results. The data just isn't there for finding optimal, and as people have pointed out elsewhere it is definitely possible to go too high.
If this does continue - and I hope it does - please check in on the effects of creatine and choline supplementation if possible. Here's Dr Gregger admitting creatine seems to have cognitive benefits for vegans (IE, may be defficient) (then dismissing supplementing it due to concerns about contaminants) https://nutritionfacts.org/video/creatine-brain-fuel-supplementation/
And for the sake of keeping relevant conversations linked, here's a mediocre questionpost I made on the EA forum about this whole question.
We have to be careful not to offset the health benefits of a vegan diet. There is a surprising amount of evidence suggesting that low-normal iron stores are beneficial and may reduce cancer incidence, mortality and perhaps increase longevity. Specifically, as strongest, I would point out the FeAst study and numerous recent Mendelian randomization studies on iron and longevity (e.g. Daghlas and Gill 2021). It is prudent to test ferritin to know whether you are too low or too high.
Vitamin D testing certainly could be useful, even though recent clinical trials testing vitamin D supplements are between somewhat to highly disappointing, but deficiency is presumably not strongly linked to veganism.
I’ve talked previously about my concerns with nutritional deficiencies in effective altruists who go vegan for ethical reasons, especially those who don’t have a lot of contact with the broader vegan culture. No one else seemed very concerned, so I launched a tiny project to test people in this group and see if they were in fact deficient. This is a report on the latest phase of the project.
To cut to the chase:
My main question here was “is there low-hanging fruit in treating nutritional deficiencies in this group, and if so how do we pluck it?” An important part of that is “how prevalent are deficiencies?”, but I had substantially more uncertainty around “do people treat deficiencies you find?” and “does the treatment lead to improvements in anything we actually care about?” That prioritization (and budget issues) led the experimental design to focus on operational issues and outcomes, and deprioritized getting the kind of clean data that would let me compare vegan and non-vegan outcomes. Similarly this write-up is mostly focused on showing the problem exists at all and building metis of investigation and treatment, rather than estimating prevalence.
Which is to say to to everyone planning on @ing me to complain about the sample size, heterogeneity, or mediocre statistics: you are right that this sample is not very informative about base rates of deficiencies in vegans or anyone else. If someone claimed it was, they would be committing an epistemic sin. However, this particular post is focused on “how much effort is it to get nutritional issues addressed, and is that effort worth it?”. Given that, any complaints about the terrible sampling will be considered to be offers of assistance in running the much larger study that could answer the prevalence question.
(if you’ve read the previous posts, this will be review)
Last year I worked in a co-working space focused on existential risks, which is anything that might really end everything. Because x-risk is a popular topic within the effective altruism movement, many participants in the space were EAs. Another big topic within effective altruism is animal suffering, especially farmed animal suffering. This has led many EAs in the office to go vegan or at least vegetarian for ethical reasons, without making animals the focus of their lives. And when I asked them, many had put no thought into how to give up animal products in a healthy way.
Which of course would have been fine if they were eating well. My personal opinion is that most people’s optimal diet contains small amounts of animal products, but lots of people are eating suboptimally for lots of reasons and I don’t consider it my problem. But the number of fatigue and concentration issues were… I don’t actually know if they were high. I don’t know what baseline to compare them to. But it wasn’t low enough to reassure me. Neither did the way the vegans talked about nutrition, and in particular the fact that more than one person was doing keto vegan and still wasn’t investigating nutrition.
None of this meant there was necessarily a problem, but I was suspicious. So I got a small grant to solve this with numbers.
First I tried broad spectrum testing, which led me to identify iron deficiency as the main concern.
Then I did a short investigation into the actual costs of iron deficiency, which motivated a number of people to get tested, some of whom got in touch after the fact.
In my last post I implied I was stopping the project, and wrote next steps as if someone else would be doing them. I ended up getting a follow on grant so pushed forward with round two on my own, which is what this post is about.
I followed the steps laid out in my previous post almost exactly.
I cannot say enough bad things about BayAreaPLS, the company we hired to come into the office and do testing.
First and most importantly, they just forgot to run the ferritin test for half the participants. They did the other tests, there was no reason to not do that one in particular, they just… didn’t. Their initial attempt to make up for this was an offer to not charge me for tests they didn’t do. I pushed back fairly hard and they agreed to some actual discounting. It’s been 2.5 months and they have yet to send me that follow-up invoice, which I guess is technically good for me but makes me feel worse overall.
Second, the results were hard to retrieve. Some participants never got results at all even after following the instructions, and I have no way to debug whose fault that is. Others didn’t bother to retrieve results because it was too hard. The happiest people were the ones who got their results over email, which is a HIPAA violation the company claims to have done by accident. Much like the lack of the second invoice, the emailed results are technically good for me but leave me more concerned about the company.
Even for people who did get results it took almost two weeks, which was not great for momentum.
Lastly the company used different deficiency thresholds for different people. They say this was based on sex and age, but they didn’t get that information from everyone (sounds like incompetence on their part), so some people got a list of thresholds. It ended up creating a moderate amount of confusion and friction, especially because I couldn’t tell what variation came from age/sex (which I mostly want to ignore) vs. the norms of a particular test (which I very much don’t). People are slow to act on results in general so the additional friction was quite costly.
Not all of these were literally caused by me, but they all fall under “my responsibility to anticipate and handle”.
I mentioned in my last post that some participants were kind of insistent on getting a lot of help from me, even after I explicitly told them something was outside my bailiwick and needed a doctor. I tried to fix that this time by hiring Aaron. That worked on that one issue, but made it harder to catch new problems. If the plan had been for people to directly show me their results and receive coaching I would have caught the missing test values much earlier. I’m not sure there is a way to have someone available enough to get all the relevant questions from participants without also having to deal with a bunch of irrelevant ones, including inappropriately persistent ones. The difference between the two just isn’t obvious with the knowledge most participants have, and emotions run so high around health stuff.
It’s hard to estimate the rate of acting on results because the people who don’t act are less likely to fill out the follow-up surveys, but my sense is it’s not good, and probably <50%. I also strongly suspect the rate of acting on results would have been higher if there had been in-person follow-up.
The guidelines I sent participants emphasized vitamin D and ferritin because it didn’t occur to me anyone could see an anemia result on the page and not rush to treat it, but at least two people scored as anemic and, as of three months later, had not treated it.
The lab obeyed HIPAA enough to only send results to participants, not directly to me, so I needed participants to forward them. Of 20 participants, 13 did so. Only 8 participated in the separate follow-up questionnaire (arguably my fault for asking for two separate forms, but there was good reason to ask for results quickly and do the follow-up a little later). 20 people with varying motivations for testing was never going to be strong evidence for anything systemic, but the low response rate makes it even harder to draw conclusions.
My emails to participants sometimes went to spam, possibly due to use of BCC, which was necessary to meet the privacy commitments I made in the application.
As a ballpark:
If you value everyone’s time roughly the same, that means that to break even we need to save one person 30 hours + ~$5700 (ignoring the information gained).
If you want to complicate that math you can add any of: discount rate on time or money, exchange rate between ops time/my time/participant time, cost of unnecessary or counterproductive treatment, knowledge gained from this round can make the next round cheaper, the fact that most of my time went to a write-up that shouldn’t really be billed to participants, but a better program would have spent more time with participants.
I should remind you here that the sample was a mix of 20 ethical EA vegans, vegetarians, people with fatigue issues, and people who happened to be in the office. Even a very large sample wouldn’t be perfectly predictive unless you had the exact same mix of participant types, and this sample was tiny. So the right way to look at these results is “is this enough to think there’s a problem?” and “did people do helpful things with the information?” not “at what exact rate does this population develop ferritin issues?”
With those caveats, here is the data from participants that reported back. I have ferritin results for 8 people and all other results for 13.
Some of the non-reporting was random due to the lab’s incompetence, but it’s not impossible unhealthy participants were more likely to report back. If you want to be extremely conservative and assume every missing value was A+ healthy, the results are still quite concerning:
But finding results isn’t very meaningful if no one acts on them. Of the 8 people who filled out the follow-up survey, 75% changed either diet or supplements, and 1 additional person kept going with supplements they would otherwise have dropped. I assume this is overreporting because people who changed things are more likely to respond, but if you assume none of the nonresponders did anything that’s still 30% of people changing something.
Of the 5 who changed something and answered the relevant question, 40% said they thought they saw an improvement (~1 month after they received results). I don’t consider that particularly strong evidence in either direction- it can take time for deficiencies to heal, but it’s also easy to placebo yourself into seeing an improvement that isn’t there. The real test will be the six-month follow-up.
Three months after testing I followed up with the two identified anemics to make sure they were getting treatment. Neither was, despite having health issues plausibly caused by anemia. They’ve both indicated vague plans to follow up now that I’ve pushed them on it.
I believe this round of testing was better than not doing in-office testing, but there is a lot of room for improvement.
My absolute wild-ass-guess is that this saved between one and ten people (out of twenty participants) from anemia or a moderate iron deficiency, and this improved their life and productivity by 10% to 100% (mean around 20%). I acknowledge these are large ranges, but some problems are bigger than other problems. I’m ignoring vitamin D entirely here because I haven’t even attempted to quantify its value and its ardent fanclub has poisoned the literature.
Even in the worst case, a 5% chance at a 10% improvement is a big deal, so I think this was obviously worth it from a participant perspective. I think it’s a toss-up if it would be worth it for apparently healthy omnivores: my expected value for them is much lower, but people don’t always realize they’re operating at a deficit and catching them requires testing actually-healthy people as well.
I said above that we needed to save someone 30 hours + $6000 for the project to break even. Even one successfully treated anemic will blow that out of the water, so I don’t feel the need to do the more complicated math with discount rates and relative value of time, especially because any future round should either be less work or have a higher response rate.
Of course “break even” isn’t a very high bar. To go even further out on a limb: the median case of mild anemia easily costs someone two hours/day (source: had anemia one time). This testing easily caught the anemia at least six months before it would otherwise have been caught (because everyone who was going to get tested on their own did so when I published my iron post). That is, at a bare minimum, 360 hours someone otherwise wouldn’t have had. That’s a pretty great return rate for 40 hours (my time + Aaron’s time + participant time) +$6000.
I would bet on yes, although a lot of information has yet to come in.
I expect this project to have 6 lasting impacts:
#1 is what I calculated above, and think was already a sound success although not resounding.
#2 and #3 require making assessments of all individuals who received testing from the project or due to blog posts I wrote. That’s a combination of “vast amounts of missing information” and “judging individual merit” that makes it really uncomfortable to talk about in a public post.
To be totally honest I’m on an “everything is bullshit” kick right now so deep in my soul I don’t think this paid off, but intellectually I think my standards are too high and this was a better project than average project in the space of existential risk.
#4 and #5 depend on other people following up on this project. I absolutely believe they should, for all the other reasons but also because the return via #6 seems pretty good- health reasons are a common reason vegans go back to eating meat [It was hard to find a good source, but this shitty source says 26%. I’ll acknowledge that’s probably an overestimate, since health is the most virtuous reason to go back to eating meat.]
I can’t estimate #6 myself. I’m not familiar enough with vegan literature to sort good from bad here, and it wasn’t my main goal.
Then there are costs. The total grant was for a little less than $25k. I didn’t track my time very closely to avoid depressing myself, but my compensation is going to work out to a fraction of my normal hourly rate. If you count foregone client work you could argue the true cost was as high as $50k.
My gut feeling is the project was straightforwardly worth it if you don’t track the foregone work. If you do, impact is dependent on having at least one of:
This study doesn’t say anything one way or the other, which means I still think yes but you shouldn’t change your opinion based on the results. The sample is too small and skewed to compare deficiency rates in vegans and nonvegans. There were energetic omnivores with deficiencies and tired vegans with perfect scores so it’s clearly not deterministic.
I see three possible follow ups to this project:
This is my default, although I don’t plan on writing many of these because there is only so much low hanging fruit and people have a very limited attention budget. I have to be very judicious in what I suggest.
Get the money to test a large enough representative sample, and run the tests with a proper control to actually estimate the cost of nutritional veganism.
This is most useful if there are EA vegan leaders who won’t act on nutritional concerns now but would if the study demonstrated a problem. If this is you, I would love to talk to you about what you would consider sufficient to act on.
Assuming the demand for this information is there, I still don’t think I want to run this project alone. First, it is a lot of work. Second…I know I said “assuming demand is there”, but I can’t picture a scenario where demand exists but no animal EAs consider this project worth working on. A collaborator would be both proof of investment and much better positioned than I am to get the information acted on.
To that end, here is an ad for a co-founder. I will post it on this blog in a few days.
This can work, but only in a limited number of situations. You need a reason (uninformed veganism, high fatigue rates) to suspect nutritional issues in lots of people sharing a space. There needs to be a reason people aren’t getting tested themselves that won’t also inhibit follow up (probably lack of money and existing relationship with a doctor). And even then it’s more of a hits-based model than a sure thing.
My decision is easy because the office I was working out of closed, and in general I think most of the people in the bay area I would want to help have already been reached. The market is saturated for at least a year. There are other offices elsewhere in the world, and if you want to run this yourself I’m happy to act in an advisory capacity (especially if you share data), but it can’t really be an ongoing project in any one city.
I finished most of this post planning on it being the end of my part of the project. I had hopes I would convince someone else to pick up the torch, and maybe even act as an advisor, but it seemed like the biggest problem was participant motivation, which I don’t feel equipped to solve. It was while I was writing this that I realized I wasn’t ready to let the broader issue of vegan nutrition go. I still believe the problem that offended my morals and epistemics is there and worth acting on.
But doing so is still very annoying, which is why I’m looking for someone to partner with on this. Someone who can handle the parts I’m bad at, point out where I’m wrong, and interface with the vegan EA community to get the results acted upon. If you’re interested, please reach out to elizabeth@acesounderglass.com.
Thank you to the Survival and Flourishg Fund for funding this research, and Lightcone Infrastructure for hosting the grant and testing. I inflicted this draft on a number of people but want to especially thank Gavin Bishop. Daniel Filan didn’t beta read this post but he did vegan-check my co-founder ad and suggest the title “I Lead This Horse To Water – You Won’t Believe What Happened Next”.