Do you have any thoughts on the risks/hazards involved here? To me that's a much more significant consideration than the price. Some thoughts / priors:
The main answer here is "see the paper"; there's a lot of discussion about this stuff. I'll summarize a few points, as I understand them, relevant to your particular thoughts.
Regarding the final paragraph, "you need some level of expertise yourself before you can distinguish real experts from fake": that has been the number one reason I didn't beat johnswentworth to the punch and post first with my experience.
I have learned more about biochemistry in the last three months than in my entire prior life combined. It has taken me three months of research, asking questions, and conferring with experts to get sufficient confidence in my understanding to commit to the project.
I'm incredibly thankful to you (johnswentworth) for posting this article; it tracks almost perfectly with my understanding, and I have no significant model conflicts with any of your observations. It raises my confidence in both my understanding, and the project, substantially.
I'd be very interested in a post on what you learned! I relied mostly on general bio background from undergrad, it sounds like you probably went into more depth in areas specifically relevant to this.
Thanks, I really appreciate you taking the time to respond.
I should probably have clarified my current views / epistemic status in my comment, since I think it sounded more skeptical than I actually am. I would say it's something like: "I expect this is quite possibly a good idea, and most probably at worst a neutral idea. I am interested in trying to elicit anything in the long tail of risks that could change that."
(I guess I did also want to encourage other people to at least briefly consider risks before trying this themselves -- although given the complexity and expense, perhaps I shouldn't worry that anybody might rush to try it.)
I think a lot of these questions are answered in the radvac paper. I sent a copy of it to a biologist I know, and asked if he thought it was crazy to do this, and he read it and said “geez this looks safer than doing drugs”. I don’t have enough expertise to add anything beyond that.
Well, this post was just crying out for some embedded predictions! So here we go:
Thanks johnswentworth for help with some of the operationalisations!
I included many different ones, as I think it is often good try to triangulate high stakes questions via different operationalisations. This reduces some some "edge-case noise" stemming from answering vague questions in overly specific ways.
@Davidmanheim you're a pretty big outlier here, and this is also the kind of question where I'd trust your judgement a fair bit:
So curious if you wanted to elaborate a bit on your model?
First, base rates are critical. Looking at potential drugs overall, the rate of approvals due to safety alone - i.e. "Investigational New Drugs" to phase-II efficacy trials, is very low. Phase 1 trials are typically 80-100 people, and most don't manage to make it past that stage. It would take much stronger evidence than I have seen to think that this vaccine is going to be outside of the norm.
Second, even if the process as done was safe, I can't imagine that greater than 99% of people manage to do this without screwing up in some serious way. That's less true of the LW crowd, but I don't think people are aware of how dumb the mistakes that get made are, or how much quality control matters, and how difficult it is with trying to enforce it for DIY projects.
Lastly, I'm well within the consensus for almost all the rest of the questions - I think it probably works in most cases, and I think it will have side effects in far fewer than 50% of cases.
(But another place I'm a bit outside the consensus is that I think it's unlikely to trigger standard antibody tests, since standard antibody tests are looking for antibodies against a specific part of the virus, and I'm unsure, reading the "Antibodies and B-cell immune response" section of the white paper, that standard tests would detect the elicited types of NABs.)
You have my admiration, and my hope that you are calculating the risks accurately!
I have not read the RaDVaC paper so I don't have a good object level model of safety and risks. From a distance it looks like heroism, because from a distance it looks like taking a risk in a way that could provide a role model for many if it works safely! It reminds me a bit of Seth Roberts who was a part of the extended tribe who did awesome stuff over and over again (seemingly safely) but who also may have eventually guessed wrong about safety.
I guess I just want to say: "This is so freaking awesome, and PLEASE be very careful, and also please keep going if the risks seem worth the benefits."
If you get a positive antibody result, have you thought about a personal challenge trial?
The big benefits to be gained from vaccination seem to be to be behavioral: going out, doing life similarly to the Before Times... which is similar to a partial/random/natural sort of "challenge trial".
I wonder if 1daysooner can or would be interested in keeping track of people who have tried the RaDVaC option, to build up knowledge (based on accidental exposures or intentional challenges) of some sort.
I double-cruxed this article because my "voice of caution" objected to it.
I eventually realized the issue was that part of my decision-making process when I do something weird, potentially risky, or expensive, is to consult with friends and family. Yet I feel that the feedback I would get from them would be so thoughtless, negative, frustrating, and potentially damaging, that it's not worthwhile. And I don't want to ignore this "consult someone first" rule, because it seems like a generally good rule that loses its force if ignored.
However, I do know some specific people who might be good to talk it over with. They're warm, open-minded, very smart, scientifically literate, unconventional, have my best interests in mind, trustworthy, and willing to discuss this kind of stuff at length. My next move is probably not to read the paper, but rather to discuss it with them.
I think what you have done here is re-invented the actual helpful version of a practice whose authoritarian bureaucratic cargo-culted version is called "anonymous peer review".
It is easy (and maybe dangerously wrong) to come to the straightforward conclusion that peer review in general is simply evil bullshit... until one finds the place from which a benevolent truth-oriented human (like oneself) finds a reason to consult with an actual "epistemic peer" as a prudent and socially-embedded response to one's own uncertainty about things one cares about.
I talked this out with a consultant friend who got his BS in biology. Here's what we came up with.
A conceptual solution would have the following variables, labeled for clarity.
Cost of vaccine = C
Probably that vaccine provides value = P
Value that could be provided per person = V
Speak of the devil. I literally just placed my peptide order a couple of hours ago. My experience (finding supplies, test runs of mixing the solution, safety profile, analysis, etc.) basically matches up with this post.
Thanks a lot for posting it.
Note: this post was frontpaged (despite a general policy of not frontpaging covid content) both because a) it seems pretty important, and b) the rationality life lessons seemed pretty timeless.
Hmm, important as in "important to discuss", or "important to hear about"?
My best guess based on talking to a smart open-minded biologist is that this vaccine probably doesn't work, and that the author understates the risks involved. I'm interpreting the decision to frontpage as saying that you think I'm wrong with reasonably high confidence, but I'm not sure if I should interpret it that way.
You should make a top-level comment about this. Chance that the vaccine works and the associated risks are object-level questions well-worth discussing.
In general, frontpage decisions are not endorsements (though I don't know Raemon's thoughts in this particular case), and this comment section is not the place for a debate about frontpaging norms. This is definitely the place to talk about chance the vaccine works and associated risks, though.
Have you run this by a trusted bio expert? When I did this test (picking a bio person who I know personally, who I think of as open-minded and fairly smart), they thought that this vaccine is pretty unlikely to be effective and that the risks in this article may be understated (e.g. food grade is lower-quality than lab grade, and it's not obvious that inhaling food is completely safe). I don't know enough biology to evaluate their argument, beyond my respect for them.
I'd be curious if the author, or others who are considering trying this, have applied this test.
My (fairly uninformed) estimates would be:
- 10% chance that the vaccine works in the abstract
- 4% chance that it works for a given LW user
- 3% chance that a given LW user has an adverse reaction
-12% chance at least 1 LW user has an adverse reaction
Of course, from a selfish perspective, I am happy for others to try this. In the 10% of cases where it works I will be glad to have that information. I'm more worried that some might substantially overestimate the benefit and underestimate the risks, however.
In my case, yes. My bio expert indicated that it was likely to be effective (more than 50%, but less than 90%) and that the risks were effectively zero in terms of serious complications.
Regarding the food grade versus lab grade question, as well as inaccuracies or mistakes in construction of the vaccine, this was a question I spent a reasonable amount of time on. The TL/DR is that the engineering tolerances are incredibly wide; the molecular weight of the chitosan isn't that important, the mixing rate isn't that important other than it be fast enough, the quantities aren't that important, exact peptide quantities aren't that important etc. A lot of these can be off by not just percentage points, but integer factors, and the result will still be acceptable.
It's also worth pointing out that unless you make serious, significant mistakes that dramatically impair effectiveness, you can always just use "more dakka" to overpower the variations. My plan is to mix each batch independently, such that at least some of the construction variations are expected to cancel. (Also, freezing the final vaccine is likely to impair effectiveness, from what little I've found on the topic.)
I wasn't sure what you meant by more dakka, but do you mean just increasing the dose? I don't see why that would necessarily work--e.g. if the peptide just isn't effective.
I'm confused because we seem to be getting pretty different numbers. I asked another bio friend (who is into DIY stuff) and they also seemed pretty skeptical, and Sarah Constantin seems to be as well: https://twitter.com/s_r_constantin/status/1357652836079837189.
Not disbelieving your account, just noting that we seem to be getting pretty different outputs from the expert-checking process and it seems to be more than just small-sample noise. I'm also confused because I generally trust stuff from George Church's group, although I'm still near the 10% probability I gave above.
I am certainly curious to see whether this does develop measurable antibodies :).
Also, having random peptides along with an adjuvant (which triggers an immune response) might be risky even in cases where those random peptides are otherwise completely safe.
There is another Covid-19 peptide vaccine developed by a Dr. Winfried Stöcker. He injected it into ≥64 volunteers, and the results he published look promising. They show both a good level of IgA, IgG and IgM antibodies and ≥ 94% neutralization for the vast majority of the test subjects. According to him (last paragraph of his blog post), none of the test subjects have reported any relevant adverse symptoms.
He describes the manufacturing in his blog (see translation below):
Man nehme dreimal 15 Mikrogramm rekombinante RBD der S1-Untereinheit (Arg319-Phe541) für eine Person. Als Adiuvans habe ich Alhydrogel von InvivoGen verwendet: Ordentlich durchschütteln und davon 200 Mikroliter mit der Tuberkulinspritze aufziehen. In eine größere Spritze 10 Milliliter Kochsalz aufziehen und die 200 Mikroliter dazugeben, mischen. Davon 500 Mikroliter pro Schuss, mit denen man seine Portion Antigen vermischt. Alles hübsch steril.
I've attempted a translation and added some of my own understanding in [square brackets]. Though I'm a German native speaker, I have zero domain knowledge in this field, so please correct me if anything is wrong:
...Take three times 15 μg [three doses of 15 μg per person, spaced
Good point! I've attempted to expand on this a bit, and list the advantages that each vaccine currently seems to have over the other:
For RaDVaC:
For Dr. Stöcker's Vaccine:
One way to achieve sterility might be to use a self-made glovebox (example tu...
Has a small community, might be easier to exchange questions and results
Given that this community exists it's likely that they somehow privately share results. It would be really interesting to know more about what's going on in that community.
as long as we knew what kind of Arg319-Phe541 peptide we need for it.
I understand Arg319-Phe541 to mean the subsection of the spike protein that begins with arginine at position 319 and ends with phenylalanin at position 541. At the moment I don't immediately find the sequence with googling but it's worth checking whether 319 is indeed arginine and 541 phenylalanin to check whether this interpretation makes sense.
The problem is that this is 222 amino acids longs which is longer then what the peptide sequencing company sell you so you can't get them the same way you get the peptide you need for the RaDVaC vaccine.
Active-Bioscience seems to sell 100 µg of SARS-CoV-2 Spike Glycoprotein-S1 RBD (319-541, biotinylated) recombinant Protein for 1.150 € which gives you enough doses for two people.
I think RaDVaC has another advantage. It's designed to be difficult for the virus to mutate to get immune to it. Having to change ...
I think deleting it was a fair response (though perhaps banning is a little over the top). assuming the moderator has no way of checking for himself whether this makes sense and he knows he doesn't, he's left with a bet about whether this is the real thing or just bullshit. he expects more bullshit than real things, and he expects the bullshit to be dangerous. so he removes everything that fits this class of things, knowing he might end up also removing something real.
It depends more on the ignorance of the moderator and on how much time he's willing to spend than on the quality of the evidence. there definitely are cases of of PHDs and maybe even professors advancing pseudoscience. so this doesn't guarantee trustworthiness.
the moderator has to make a decision in a state where he can't trust himself to distinguish real stuff from bullshit. he goes for minimizing harm at the cost of deleting novel good ideas. seems like a sensible decision to me.
Often, e.g. Stanford profs claiming that COVID is less deadly than the flu for a recent and related example.
Via Sarah Constantin's Twitter:
I looked into this, because yay citizen science. I could not find one research study using any of the peptides in the RADVAC white paper that found they inhibited SARS-CoV-2 infection in cells, let alone animals or humans.
and
“Take a random peptide that has never been tested on any living thing” is not at all the same thing as “take a well-known, well-studied recreational drug”, as far as risk goes.
She doesn't explicitly state that this has never been tested on any living thing. Possibly because she wasn't confident enough in her research survey to claim that, possibly because she was drawing a starker contrast than applies to this instance. But all the COVID testing for RADVAC is purely in silico, so while the chemicals involved may be studied for safety in vivo, efficacy is completely untested even at the (much simpler than organism) cell level.
So the EV of the benefits are low, and the risks are unclear.
Related: This was discussed on LW in August 2020, someone claims to have done it in December: https://www.lesswrong.com/posts/62WuBbQpSwAbctGDP/what-price-would-you-pay-for-the-radvac-vaccine-and-why
Vaccines that are brought to clinical trials have a 33.4% approval rate, which seems like a reasonable estimate of the chances that this vaccine works if executed correctly. Note that this is from trials conducted from 2000-2015.
I probably have a roughly 5% chance of catching COVID before I'm vaccinated. Given my age, COVID would put me at a 0.2% risk of death. Let's double that to account for suffering and the risk of long-term disability.
If I value my life at $10,000,000, then an intervention that gives me a 33.4% chance of avoiding a 5% chance of a 0.4% chance of death is worth $668. So it seems like I'd want to be vaccinating at least one other person in order for this to be worthwhile.
I welcome any further thoughts on this expected value calculation. In particular, I think it's possible that I'm dramatically underestimating the risk and potential severity of long-term symptoms. It doesn't take much additional risk to make this project worthwhile for a single person.
Regarding the 33.4% approval rate: based on what I've learned about traditional vaccine development and production in the last few months, I am not at all surprised. Both peptide and RNA vaccines are effectively "state of the art" technologies compared to traditional vaccine techniques. It's like comparing modern non-invasive out-patient surgery to the 1970's equivalent.
You need look no further than the russian and chinese vaccines - those use the rather crude technology of "throw big chunks of inactivated virus particles at the immune system and hope that the immune system guesses the right antibodies to deal with the live version."
Both peptide and RNA vaccines are instead, "we have identified very specific antibodies which we know are effective both from the serum of recovered patients and from computational modeling, then use exactly the minimal protein sequences needed to generate those antibodies."
Both the russian and chinese vaccines use chunks of proteins that are thousands (and likely tens of thousands) of amino acids long, in a mostly inactivated form. The immune system has no idea what to latch onto, what will be effective at stopping replication, ...
A lot of people have been working really hard for the last year to discover, understand, and know these things. It's the foundation for how the mRNA vaccines work.
Perhaps take a look through this:
https://www.sciencedirect.com/science/article/pii/S2319417020301530
Wow!
I guess a thing that still bugs me after reading the rest of the comments is, if it turns out that this vaccine only offers protection against inhaling the virus though the nose, how much does that help when one considers that one could also inhale it through the mouth? Like, I worry that after taking this I'd still need to avoiding indoor spaces with other people, etc, which would defeat a lot of the benefit of it.
But, if it turns out that it does yield antibodies in the blood, then... this sounds very much worth trying!
Back in December, I asked how hard it would be to make a vaccine for oneself. Several people pointed to radvac. It was a best-case scenario: an open-source vaccine design, made for self-experimenters, dead simple to make with readily-available materials, well-explained reasoning about the design, and with the name of one of the world’s more competent biologists (who I already knew of beforehand) stamped on the whitepaper. My girlfriend and I made a batch a week ago and took our first booster yesterday.
This post talks a bit about the process, a bit about our plan, and a bit about motivations. Bear in mind that we may have made mistakes - if something seems off, leave a comment.
The Process
All of the materials and equipment to make the vaccine cost us about $1000. We did not need any special licenses or anything like that. I do have a little wetlab experience from my undergrad days, but the skills required were pretty minimal.
The large majority of the cost (about $850) was the peptides. These are the main active ingredients of the vaccine: short segments of proteins from the COVID virus. They’re all <25 amino acids, so far too small to have any likely function as proteins (for comparison, COVID’s spike protein has 1273 amino acids). They’re just meant to be recognized by the immune system: the immune system learns to recognize these sequences, and that’s what provides immunity.
The peptides were custom synthesized. There are companies which synthesize any (short) peptide sequence you want - you can find dozens of them online. The cheapest options suffice for the vaccine - the peptides don’t need to be “purified” (this just means removing partial sequences), they don’t need any special modifications, and very small amounts suffice. The minimum order size from the company we used would have been sufficient for around 250 doses. We bought twice that much (9 mg of each peptide), because it only cost ~$50 extra to get 2x the peptides and extras are nice in case of mistakes.
The only unusual hiccup was an email about customs restrictions on COVID-related peptides. Apparently the company was not allowed to send us 9 mg in one vial, but could send us two vials of 4.5 mg each for each peptide. This didn’t require any effort on my part, other than saying “yes, two vials is fine, thankyou”. Kudos to their customer service for handling it.
Besides the peptides, all the other materials and equipment were on amazon, food grade, in quantities far larger than we are ever likely to use. Peptide synthesis and delivery was the slowest; everything else showed up within ~3 days of ordering (it’s amazon, after all).
The actual preparation process involves three main high-level steps:
Prepping a batch mostly just involves pipetting things into a beaker on a stir plate, sometimes drop-by-drop.
Finally, a dose goes into a microcentrifuge tube. We stick the intake tube of a sprayer into the tube, and inhale.
That’s the process, at a high level. Multiple boosters are strongly recommended, so there’s a few iterations of this, though only the “take stuff out of the freezer and mix it together” step needs to be repeated. See the whitepaper for the full protocol details, as well more information about each of the peptides and what the other ingredients do (summary: chitosan nanoparticles).
The Plan
The key problem is how to check that the vaccine worked. If it were injected, that would be easy: just get a standard COVID antibody test. Inhaling makes it a lot harder to hurt yourself, but also complicates testing.
The whitepaper goes into more detail and half-a-dozen different types of immune response, but the basic issue is that immunity response in the mucus lining (i.e. nose, lung, airway surfaces) can occur independently of response in the bloodstream. Commercial COVID antibody tests generally check a blood draw. In principle one can run a similar antibody test on a mucus sample, but <reasons>, so the commercial tests check blood.
(Side note: in many ways immunity in the mucus lining is better than in the blood, since it blocks infection at the point where it’s introduced. This is an advantage of inhaled vaccines over injected. So why do most commercial vaccines inject? Turns out logistics are a major constraint on commercial vaccine design, and injections are surprisingly easier logistically. One of the major relative advantages of radvac is that it’s intended to be prepared on-site shortly before administration, so it can use techniques which work better but don’t scale as well. That largely balances out the constraints of readily-available materials and simple preparation. As usual, the whitepaper goes into much more detail on this, including several other logistics-related relative advantages - multiple boosters, custom peptides, frequent design updates, etc.)
The whitepaper claims that “over a hundred” researchers have self-administered the vaccine so far, but I have not been able to find any data on test results from any of them. The paper says that inhaled vaccine can induce immunity in the blood, but I don’t have a quantitative feel for how likely that is, other than the usual assumption that more dakka makes it more likely. Meanwhile, I don’t have a convenient way to test for immune response other than the commercial tests.
So, the current plan is to search under the streetlamp. We’ll just use the commercial tests. Both of us got an antibody test before starting the project, and both came back negative.
My current model is:
So, we’ll do (up to) two more blood tests. The first will be two weeks after our third (weekly) dose; that one is the “optimistic” test, in case three doses is more-than-enough already. That one is optimistic for another reason as well: synthesis/delivery of three of the nine peptides was delayed, so our first three doses will only use six of them. If the optimistic test comes back positive, great, we’re done.
If that test comes back negative, then the next test will be the “more dakka” test. We’ll add the other three peptides, take another few weeks of boosters, maybe adjust frequency and/or dosage - we’ll consider exactly what changes to make if and when the optimistic test comes back negative. Risks are very minimal (again, see the paper), so throwing more dakka at it makes sense.
Consider this a pre-registration. I intend to share my test results here.
Motivations
Why am I doing this?
I imagine, a year or two from now, looking back and grading my COVID response. When I imagine an A+ response, it’s something like “make my own fast tests, and my own vaccine, test that they actually work, and do all that in spring 2020”. We’ve all been complaining about how “we” (i.e. society) should do these things, yet to a large extent they’re things which we can do for ourselves unilaterally. Doing it for ourselves doesn’t capture all the benefits - lots of fun stuff is still closed/cancelled - but it’s enough to go out, socialize, and generally enjoy life without worrying about COVID.
I’ve written a blog post about Benjamin Jesty, the dairy farmer who successfully immunized his wife and kids against smallpox the same year that King Louis XV of France died of the disease. I explicitly use this as an example of what Rationalism should strive to consistently achieve. Yet when a near-perfect real world equivalent came along, on super-easy mode with most of the work already done by somebody else, it still took me until December to notice. The radvac vaccine showed up in my newsfeed back in July, and I apparently failed to double-click. That level of performance is embarrassing, and I doubt that I will grade my COVID response any higher than a D.
So I’m doing this, in part, to condition the mental motions. To build the habit of Doing This Sort Of Thing, so next time I hopefully do better than a D.
Of course, the concrete benefits are nice too. But at this point it’s only ~4 months until I’d get a vaccine anyway, so the price tag is only arguably worthwhile. It’s still a fun project in its own right, and it gets dramatically cheaper with more people (remember, $1000 bought enough supplies for ~500 doses). Concretely, the largest benefits are in risk reduction. If there’s big hiccups in commercial vaccine deployment, this becomes much more worthwhile. If the South Africa strain turns out to evade commercial vaccines, this becomes much more worthwhile - the radvac design is frequently updated based on the latest COVID research, so we hopefully wouldn’t need to wait around for approval of a new commercial vaccine.
Finally, I'm curious whether it will work - or whether we'll be able to tell that it works. It's a data point as to just how often large bills are left sitting on sidewalks just a little ways off the beaten path.