Another RadVac Testing Update

by johnswentworth5 min read23rd Mar 202117 comments



Previously: Making Vaccine, Commercial Antibody Test Results, Mini-Update

I've now run 9 ELISA tests. The main result is noise: negative controls are all over the map, sometimes very blue (i.e. positive), sometimes not blue at all. I did see more positive results in the experimental group than I'd expect from noise alone, but I haven't gotten the noise to a point where results are consistently reproducible.

Meanwhile, I also ran one very simple test: I snorted a batch of the peptides, without the chitosan or anything else - just peptides in deionized water. Previously, on doses 3-6 of the vaccine, I had consistently been congested for a couple days after (and not congested the rest of the week), which strongly indicates an immune response. However, that response could have been to the chitosan or other contents of the vaccine, rather than the peptides. This test put that possibility to rest: after snorting just the peptides, I was very obviously congested for a couple days, in basically the same way as after the vaccine doses.

So thanks to that simple test, I personally am now pretty highly confident that I have an immune response to these peptides. Unfortunately it's not as legible as an ELISA test, so you should not necessarily be quite as convinced by this.

Now, this still leaves the question of whether an immune response to these peptides translates into an immune response to COVID. It could be that e.g. the conformation of the peptides' corresponding sequence within full COVID proteins is different enough that it doesn't carry over. Personally, though, I consider this a much less likely failure mode, for two reasons. First, the white paper indicates that the peptides were chosen based on antibodies developed by people who actually had COVID. Second, whether antibodies against these peptides bind the real proteins is something which I would not expect to vary much from person to person, so if it's worked for a few people it should work for everyone - and the whitepaper does indicate that multiple groups have seen positive results testing for binding against the full proteins.

None of this puts my confidence up close to 99%, but I'm now considerably more confident that the vaccine worked (~90%). Also, that confidence is distributed over fewer possible worlds - e.g. based on the info in the latest version of the RadVac whitepaper, I now very much doubt that the vaccine will induce a response in the blood (unless it's injected). I also now have very little weight on the possibility that it works for some people sometimes but didn't work for me specifically, so additional dakka is not needed (at least for me).

The next section will be a bit more detail on the ELISA tests, for people who are curious about exactly how that sausage was made.


This section is an abbreviated chronology of what-I-saw and my reasoning about it; it's intended to show how I came to the conclusions I did. I expect most people will not find it very interesting, but one of the benefits of blog posts is that I can show all the questionable decisions and opportunities for confirmation bias to sneak in, so that's what I'm doing.

First, some background on how these tests work in theory. We start with a "high binding plate" - basically some plastic treated so that proteins/peptides stick to it.

That’s the plate; each of the holes is called a “well”, and is basically a mini-test-tube with a high-binding surface.

We add a solution of our peptides, and some of them stick to the surface. Next, we dump that solution out, leaving behind only the peptides which bound to the surface. We add some "binding solution" - in this case nonfat dry milk with a little detergent in it. The proteins in the binding solution fill whatever space on the surface was not taken up by the peptides.

Now, with the foundation in place, we build a tower. We add a nasal wash sample from my nose, which hopefully contains antibodies that bind to the peptides. Then we dump that out, leaving behind only the antibodies which bound to a peptide attached to the plate. Next we add the "secondary antibodies", which bind to my antibodies and have an enzyme attached to them. Then we dump that solution out too. If all goes well, this leaves our "tower": peptide bound to the plate, antibody bound to the peptide, secondary bound to the antibody.

The final step is to add some TMB solution. The enzyme attached to the secondary antibody will turn the TMB blue, so if we see blue after a few minutes, then we know the secondary antibodies are present, and hopefully that means the rest of the tower is present too.

So how does this play out in practice?

Here's the first plate I ran: 

Each row is a different sample (nasal wash and saliva from both myself and my girlfriend, blood from me, and one more concentrated nasal sample). Each column is a different peptide, with no-peptide control group in the right-most column. From this, we learn three things. First, there’s a lot of variation between samples. Second, the variation we’re seeing has nothing to do with the peptides. Third, and most important, the negative controls contain an awful lot of blue. So this does not match our theoretical picture of building-a-tower; somehow, something in the sample is sticking in the plate, and it’s sticking to something other than the peptide.

Next came a fair bit of trying stuff. I tried both more concentrated peptides, and dosing myself with RADVAC the day before taking a sample in order to induce more antibody production, both in hopes of increasing the signal enough to overcome the noise. Results were basically similar. I tried a bunch of different blocking agents, without any peptides at all, to see which gave the least-blue negative controls - dried milk was actually one of the best, although egg whites were better. So I tried using egg whites for the blocking solution, and results were basically similar. I ran another plate just comparing various negative controls, without any major new insight.

In general, I still saw some samples produce generally-more or generally-less blue, across both experimental and negative control groups, for no clear reason. Generally, things varied more from test-to-test and sample-to-sample than within similar treatment groups on the same sample/test - i.e. the noise is mostly systematic.

The main result from all this was that wells with no sample were pretty consistently not blue. Whatever causes my no-peptide negative controls to turn blue, it definitely involves something in the sample binding to something other than the peptide.

I did some reading online and some thinking about how to reduce the sample binding in the negative control group. I shifted to a mental model where a significant fraction of the peptide/binding agent on the plate was actually coming loose and being replaced by whatever was in solution. One way to reduce noise in the control, then, is to include a little binding agent in the antibody solution and secondary antibody solution - so if a space on the plate opens up, it will most likely be filled by the extra binding agent rather than the antibody. (This was included in the protocol, but listed as “optional”, and I didn’t understand before why it would be useful.) I also increased the amount of binding solution used (so that it covered the sides of the well more completely), and made extra sure to not accidentally use 100uL rather than 200 uL of binding solution (all of the other steps involve 100 uL in the well, so it’s an easy mistake to make if not paying attention - I think I probably made this mistake multiple times in earlier tests). Finally, I removed the wash step between peptide and binding solution - I see no way that that one could reduce noise, and I’d expect it to reduce the signal.

With all that in place, I ran two more tests. One saw generally very little blue, but there was more blue in the experimental wells than the controls:

Left-to-right, we have three no-peptide controls, then three experimental wells, then three more no-peptide controls, then three more experimental wells. There is definitely some blue in the no-peptide controls (especially the second well from the left), but there is generally more blue in the experimental wells. Note that this test was run with a sample which had been frozen; I think that’s the most likely cause of the generally-faint blueness.

Then we got the most promising result:

The group of five experimental wells in the upper right was visibly more blue than the corresponding no-peptide negative controls below them. (The group on the left is a sample which was frozen; so now we know to use fresh samples. This is another mistake I made multiple times in earlier tests.)

On the other hand, I’m still not able to get consistent results. I ran one more test the next day, just a single fresh sample with a whole bunch of experimental wells and no-peptide controls, and there was no visible difference between the controls and the experimental wells:

Experimental and control wells are in alternating rows. The last two columns are no-sample controls.

So, bottom line: the two positive results, especially the second, are more than I’d expect to see from noise after having run this many tests. But they’re still definitely not knock-down unambiguous evidence, and there’s still a lot of test-to-test variability which I’m unable to account for.

Going Forward

Mainstream vaccines will be available to the general populace here starting two weeks from today, so this project is probably reaching its end soon. We may run another test or two, but I probably won't have another post. Conditional on any more tests, I will put up a shortform summarizing whatever results we see.


16 comments, sorted by Highlighting new comments since Today at 10:20 PM
New Comment

Do you have any thoughts on what you'd do differently to be more personally confident doing this again?

Good question.

One thing I looked into was obtaining fast antibody tests (basically strips of paper with some proteins and colloidal gold soaked into them). They're "research use only" and hard to get your hands on if you're in the US, but if you're outside the US (or have a friend outside the US willing to help) it should be easier. They would make it dramatically easier and less error-prone to test, and they'd also test for binding against full COVID proteins directly (rather than the radvac peptides). If I were going to invest much more effort into this project, I'd try harder to obtain some - either find some international seller who doesn't ask a lot of questions, or recruit a friend who lives outside the US and can get their hands on some.

Another thing I might try is buying a microcentrifuge. A really low-end minimal one costs under $200, so it wouldn't increase the relative cost of the project too much. The main use would be to clean up the samples - i.e. it could remove most of the sticky crap from a nasal wash sample. If noise is coming from that sticky crap sticking in the plate, then removing it would help a lot. If.

Yet another thing I might try is getting a COVID ELISA kit. This has similar considerations to the fast antibody tests.

They’re “research use only” and hard to get your hands on if you’re in the US, but if you’re outside the US (or have a friend outside the US willing to help) it should be easier.

I'm willing to help.

Snorting peptides directly is hilarious! I should do lines of peptides at the next corona party :)

Theoretically, it shouldn’t cause an immune response, as peptides shouldn’t be immunogenic on their own, that’s why you need chitosan as a delivery mechanism and adjuvant. However, who knows? Was it actually researched and proven that peptides on their own do not cause an immune response no matter how big is the dose and route of administration? I could well imagine that this is simply a theoretical conclusion that was never empirically verified, or that it was only verified by an injection, but not by snorting, and peripheral immune system is triggered by pure peptides, while systemic is not, or that it was only tried in much lower doses. Even if a 10-100 times higher dosage of peptides is equivalent to chitosan+peptides, this is likely of little commercial interest as chitosan “enhancement” is cheaper and more scalable at commercial scale than peptides production. So it might actually work and it is cool that you’ve tried it and there is evidence that “you were congested for a few days like after previous vaccine applications and it looks like nothing really bad had happened”.

What dose of pure peptides did you use per 1 peptide and totally? The currently recommended dose is ~200ug in total, no matter how many component peptides are there. If you do 5 peptides like in the v10, then  this is 40ug per peptide. If you do 1mg=1000ug of pure peptide, then this is 25 time more dakka and it might well trigger the same response as 40ug of peptide+chitosan.

I think the currently recommended does is likely to be too low for average human risk preferences. Organisations are extremely risk averse, the negative side effects are given much more weight than unrealised benefits, so the recommended dose is likely to be “the smallest one that kind of works without any side effects” rather than the optimal one. There is almost no evidence for the efficiency of different doses and this is all guesswork based on guesswork of others in slightly related cases, e.g. the 500ug cancer peptide vaccine dosage.

Congestion after vaccine/peptides administration could be due to many factors, so I consider it only a very weak evidence. This can be purely psychosomatic. The vaccine is acidic, so it irritates your nasal mucosa. When you put anything in your nose, it may irritate it and cause congestion. You have applied pure peptides after receiving several doses of vaccine and after enough time has passed for developing an immunity, so congestion may indeed be an immune response, but it may only happen when you already have an immunity, but it won’t be triggered if you have no immunity. Actually, this is what your data suggests. “Previously, on doses 3-6 of the vaccine, I had consistently been congested for a couple days after”. I understand this as you having no congestion after doses 1-2 and getting it after all further doses. If this is so, then this looks like the actually important immune response of a naive immune system happens without congestion, but the immune response of a trained immune system happens with congestion.

Theoretically, it shouldn’t cause an immune response, as peptides shouldn’t be immunogenic on their own, that’s why you need chitosan as a delivery mechanism and adjuvant.

It shouldn't cause an immune response if you snort them without having snorted peptides + chitosan previously. If you have immunity because you snorted both of them previously I would expect a immune response. 

Yes, I meant immune response from a naive individual, I didn't know what happens for an immune individual. If so, then the congestion pattern makes sense and is evidence for immunity.

This can be an interesting test: snort peptides and interpret congestion as immunity.

Do you know if congestion is due to antibody or cellular immunity or both?

Dosage was the same as the (gen 9) vaccine: ~120 ug total, divided by 9 peptides.

The point of the exercise was to test whether I had acquired an immune response, not to act as a vaccine, so your last paragraph is on-point.

This test put that possibility to rest: after snorting just the peptides, I was very obviously congested for a couple days, in basically the same way as after the vaccine doses.

That seems pretty strong evidence to me.

I would do a placebo control too, just to make sure.

I would do a placebo control too, just to make sure.

My prior that snorting DI water would do nothing was pretty strong, but I had intended to test it anyway, so thanks for the reminder.

I snorted some DI water last night, in the same manner that I snorted vaccine/peptides. With the vaccine/peptides, I pretty consistently woke up congested the next morning, and blew my nose every few minutes throughout the day. None of that has happened with the DI water - it's just been a normal day so far, in terms of congestion.

I'm not sure DI water would be a suitable "placebo" here. Perhaps a placebo effect is not even what is occurring. Previously you were inhaling something with small particles -- a bit like what happens every spring with pollen. Perhaps a test with some other inert matter that might not even be able to invade your body much less produce some type of chemical reactions with the cells or cellular processes?

Yeah, that had occurred to me too. If people have other suggestions for what to snort, I'm open to ideas. Though my priors are still pretty strong here - I've accidentally snorted enough things to know that most things don't induce significant congestion.

Ok this may be a naive question, but given that John brews the stuff and expects certain results anyway: Isn't being congested something that might simply follow from actual placebo effects?

That is definitely something to worry about. I would be very surprised if the degree of congestion I experienced could be induced by a placebo effect, but this is exactly the sort of reason why I said the result lacks legibility - the rest of you haven't been able to see exactly what the symptoms looked like.

Effective blinding is definitely more involved if one's both experimenter and test subject.  It's not impossible but an assistant would help a lot.  Controlling for placebo effect does seem one of the big issues at this scale.

I could prep both some peptides and a control and have my girlfriend randomly pick one, although the peptides do have a detectable scent to them, so I don't think this would be enough to actually blind me.

You could add some other scented ingredient to both peptide and control solutions. Rosewater would be a pleasant option. I wouldn't expect this to interfere with any immune responses too much, but you should do some research to check if you decide to try this.