It is always good to have additional data, but this result is totally expected, so I don’t update much based on it. When I’ve discussed the efficiency evidence with RaDVaC team, they mentioned that:
- they have tried commercial tests and didn’t manage to get even one positive result
- they have tried ELISA to measure the blood antibody titers and got some good results (part awesome, part good, part not so good)
- they have tried ELISA to measure the saliva antibodies, there are more of them than in blood samples, but there is no methodology to translate the raw data into titers for saliva
- they focus on B-cell target mechanism, not ACE2 binding and definitely not serum antibodies, so they don’t care that much about the serum antibody titers
Do you know whether the ELISA tests were against the radvac peptides specifically, or against full proteins?
- they have tried ELISA to measure the blood antibody titers and got some good results (part awesome, part good, part not so good)
Have you asked why that resulting data isn't public? Is it fear of legal issues of being accused to run clinical trials?
I haven't. Firstly, there is no proper data, just some bits of evidence. Secondly, yep, I am pretty sure that they would get in trouble if they did anything that looked like a trial, so I assume that they stay on the safe side and well, don't do anything like a trial.
Curated.
The previous post, Making Vaccines post was one of the top-upvoted posts of all time. I think it's great for LW-folk to actually go off and do Actual Science™, and then even better to follow that up with reporting on the results, and then even better to report negative results, and then even better still to list out their thoughts on what those negative results actually mean and how to continue reasoning about the domain.
I remain somewhat uncertain how to think about the RadVac vaccine in particular, but I think both this post and the previous one are great to read from a standpoint of "orienting towards science" – seeing the world as something we can understand, and that we're not only allowed to reason about, but allowed to go out and Interact With The Physical World in order to learn more about.
I agree that posting the results was the correct thing to do, and appreciate that John is trying to figure out if this is useful - but I actually claim the post is an example of how rationality is hard, and even pursuing it can be misleading if you aren't very, very careful.
In The Twelve Virtues of Rationality, this post gets virtue points for the first (curiosity, for looking into whether it works,) third (lightness, being willing to update marginally on evidence,) fourth (evenness, updating even when the evidence isn't in the direction desired,) sixth (empiricism, actually testing something,) and tenth (precision, specifying what was expected.) But virtue is certainly not a guarantee of success, even for completely virtuous approaches.
I think this tries to interpret data correctly, but falls short on the eleventh virtue, scholarship. For those who want to do Actual Science™, the first step is to know about the domain, and make sure your experiment is valid and useful. Going out and interacting with reality is valuable once your models are good enough to be able to interpret evidence. But Science is hard. (Perhaps not as hard as rationality, at least in some ways, but still very, very difficult.) In this case, without checking what the specific target being tested for was, as Christian notes, the data doesn't actually provide useful evidence. And if he had a (recent) asymptomatic case of COVID, the result would have been positive, which is evidence that the vaccine doesn't work, but would have been interpreted as evidence that it did.
They did test fot antibodies before taking RadVac. That was a prrcaution to lower the probability, that they would get antibodies due to infection.
I'm not sure why there's updating based on the fact that peptides from proteins besides the spike protein don't result in antibodies agains the spike protein being build. It seems to me a faily straightforward prediction. I don't see how antibodies against the spike protein would be developed without those peptides.
You need to see if the spike peptide included corresponds to the antibody being tested for - and given how many targets there are, I would be surprised if it did.
Despite holding a far lower prior on efficacy, I'm agreeing with Christian - this evidence shouldn't be a reason to update anywhere nearly as strongly as you did against effectiveness.
Based on the fact sheet, it sounds like they're using full S1 and S2, which together constitute basically the entire spike protein. You seem to be imagining that they would use short peptides in the test rather than whole proteins or large subunits; any particular reason why?
Not sure that you'd get reactions from large subunits if they fold differently than the full spike - but my biochemistry/immunology isn't enough to be sure about how this would work.
The marketing material for the test kit has a long description of steps they took to get the same conformation. It's still marketing material, so I don't trust it 100%, but there's at least a plausible story that it should be the same.
Strong upvote even though this is kind of a meh update, because I want to encourage actually gathering the data and reporting on it.
I also strongly upvoted for the same reasons. Very much looking forward to the results of the ELISA mucus test!
"Aside from the test result, we do have one more small piece of information to update on: I was quite congested for 1-2 days after the most recent three doses (and I was generally not congested the rest of the week). That's exactly what we'd expect to see if the vaccine is working as intended, and it's pretty strong evidence that it's doing something."
Agree that this is evidence it is doing something, but my strong prior is that the adjuvant alone (chitosan) would cause this to happen.
I'm also unclear about why you chose the weekly schedule, or if you waited long enough to see any impact. (Not that the RadVac test would tell you anything.) The white paper suggests *at least* one week between doses, and suggests taking 3 doses, for healthy young adults.
According to the white paper, you're likely to be protected, and I think continuing now would add danger without corresponding benefit. You said in the original post that you might continue dosing. I don't know enough to comment usefully about either immune tolerance or adjuvant hyperstimulation, but I suggest talking to a immunologist about those risks and how they change if you in fact continue and try "more dakka," since continuing to does seems like it would increase those risks.
Strongly agree that ELISA tests are more valuable than more RadVac, and it would be at least moderate evidence one way or another. (But even if you can induce immune reactions to parts of the virus, it' unclear how much that would actually reduce your risk if infected.)
My current plan is to do one more dose, to make sure we've had three total doses with all nine peptides, then cut it off there. In general, I expect more dakka to be valuable mainly if the dosage is sufficient to induce response in some people but not everyone, so e.g. if we saw a response in my girlfriend but not in me then I'd be a lot more inclined to add another couple doses.
I'm generally happy to see someone do something concrete and report back, and this was an exceptionally high-value thing to try.
Background: Making Vaccine
Results are in from the commercial antibody tests. Both my girlfriend and I came back negative - the test did not detect any Spike antibody response in the blood. This post will talk about how I'm updating based on these results, and the next steps.
Here's our timeline so far; more info on the vaccine is in the original post and the radvac whitepaper:
The blood draw for this test took place the day after the fifth dose. I expect this is too soon to notice significant impact from the last two doses; vaccines in general seem to typically take 2-3 weeks to kick in, and that is my expectation for this one as well. (Also, it was an "IgG antibody test", and WebMD says these antibodies typically take about 2 weeks to show up after covid symptoms show from an actual infection.) This is intended to mainly be a test of the first three doses.
The test apparently used the "DiaSorin Liaison(R) SARS-CoV-2 S1/S2 IgG assay" (I didn't know this until the results came in). According to the FDA, it has about 92% sensitivity and 99% specificity. The "S1/S2" part indicates that it's testing for response to the S1 and S2 subunits of the spike protein - together, these are essentially the whole spike protein.
Important thing to notice: the test was looking for Spike antibodies, and two of our three missing peptides were Spike peptides. Indeed, there were only 3 Spike peptides among the full 9, so with two missing, we only had one Spike peptide in our first three doses. (The rest target other parts of the virus.) So that makes the test significantly less useful than it would otherwise be, and makes me more inclined to get another test in 2-3 weeks when the doses with the other three peptides have had time to kick in.
How I'm Updating
In the original post, I called this test "searching under the streetlamp". It wasn't super likely to come back positive even assuming the vaccine worked as intended, but it was relatively cheap and easy to run the test, so it was our first check. Given the missing Spike peptides and the test only checking against Spike, it was even more likely to come back negative than I originally estimated.
In Jacob's prediction questions, I gave roughly a 25% chance that a commercial antibody test would pass for most people, given three doses and all 9 peptides. I gave the vaccine about 75% chance of working overall, distributed over several different possible worlds. In this specific scenario, it's clear that the prior on test passing should be even lower.
(Reminder on the possible worlds: the vaccine could induce antibody response in the blood and mucus, only mucus, or not at all. It could induce T-cell response separate from antibody response. It could work sometimes, much like how the first dose of commercial mRNA vaccines tend to work in 75% or 85% of people, and in that case I expect more doses/more time to make it work more often.)
After updating on the results, I'm down to about 60-70% chance of working overall. Unfortunately this test just didn't give us very much information - at least about the vaccine working.
Aside from the test result, we do have one more small piece of information to update on: I was quite congested for 1-2 days after the most recent three doses (and I was generally not congested the rest of the week). That's exactly what we'd expect to see if the vaccine is working as intended, and it's pretty strong evidence that it's doing something. Updating on both that and the test results, I'm at ~70% that it works overall.
Next Steps
There's a few directions to go from here.
First, we can take one more dose with all the peptides included, wait a couple more weeks, and get another test to see if there's a blood antibody response against the Spike protein. That would still be searching under streetlight - it's still cheap and very useful if it passes, but most likely to not pass even if the vaccine works.
We could also specifically look for a commercial test which checks for both Spike and Nucleocapsid antibodies, rather than just Spike, but I expect that to be difficult - those sorts of details are not particularly visible to consumers. Also, it would still most likely be a blood test.
The most interesting direction to go from here is to order an ELISA assay kit and run a test ourselves. Official Best Person Anna Czarnotta suggested a protocol and gave a bunch of helpful tips on this in the comments on the previous post. This would not require waiting another 2-3 weeks, and would allow us to test mucus directly. It would be a pretty direct test of whether the vaccine works, not a searching-under-the-streetlight test. So that's probably what we'll try next; I need to do some reading to figure out the details. Further results will be posted when they come in.