The U.S. CDC has released new guidelines on antibody testing that note it’s possible for a test that’s 90-95% accurate to deliver incorrect positive results more than half the time. That sounds shocking, but it can be true—and it’s important to understand why, if you’re considering getting tested or have already gotten a positive result.
In this post I’m talking about coronavirus antibody testing, specifically. Although the caveats we’ll discuss apply to many types of testing, they’re especially important as we try to figure out how many people have been exposed to the new coronavirus.
There are different ways to measure accuracy
How do we know how accurate a test is? We have to measure it against another test. For antibody tests, accuracy numbers come from the following:
How many positive samples does the test correctly identify as positive? For this, labs use blood samples from people who were known to have a COVID-19 infection and who tested positive with a swab test. If a test correctly identifies 90% of these as positive, we say the test has 90% sensitivity.
How many negative samples are correctly identified as negative? To answer this question, labs use samples collected in pre-pandemic times, before anyone could possibly have been infected with the virus that causes COVID-19. If the test correctly identifies 95% of the negative samples as negative, we say it has 95% specificity.
Those sound like pretty high numbers—90 and 95 per cent, in our example—but they don’t answer the question of what a test result means. (I chose these numbers because they’re what the CDC uses in their example, by the way.) For that, we need to know how common the virus is. Yes, this is a little bit of a catch-22, since the only way to find out is to test people. But with the accuracy rates as calculated above, it’s possible to come up with estimates. We know we’re nowhere near herd immunity, for example, and more likely somewhere around 5% in many locations.
What does a positive result mean?
Let’s say 5% of people in your community have developed antibodies to the coronavirus. A nice representative 100 of you line up at a testing site. Five of you truly have the antibodies, and 95 do not.
Of the five people with antibodies, 90% are correctly identified as having them. That’s 4.5 people, so let’s round up and say all five get positive results. So far, so good.
But of the 95 without antibodies, only 95% are correctly identified as negative. That means 90.25 people (let’s round to 90) get negative results, leaving five who get positive results.
Who walks out of the testing centre that day? 90 people with negative results, and 10 people who just got told they are positive.
Of those 10, 5 are true positives and 5 are false positives.
There’s a name for the number we just calculated: the positive predictive value. This test only has a positive predictive value of 50%, because true positives are so rare. (If, instead, 70% of people had actually been exposed to the virus, there would be 63 true positives, and one or two false positives.)
What should I do if I have a positive test result?
Here’s the problem: Anecdotally I’ve heard of people getting positive antibody results, and then figuring that they are immune to the disease. They visit friends, they walk around maskless, they assume there’s no harm in taking jobs that would put them in close contact with infected people.
But our tests just aren’t good enough to use as a basis for those kinds of decisions. In fact, the CDC says that you shouldn’t do anything differently after getting a positive test result.
Asymptomatic persons who test positive by serologic testing and who are without recent history of a COVID-19 compatible illness have a low likelihood of active infection and should follow general recommendations to prevent infection with SARS-CoV-2 and otherwise continue with normal activities, including work.
There should be no change in clinical practice or use of personal protective equipment (PPE) by health care workers and first responders who test positive for SARS-CoV-2 antibody.
Briefly: if you weren’t sick, you shouldn’t assume you ever were. And if you were sick, you should still follow whatever guidance you were given before. The test changes nothing for you personally.
Antibody tests are useful for large groups of people, so that a mayor or governor or epidemiologist can have an estimate of how common infections truly are. Research on this is ongoing. They’re not useful for decision-making on an individual level. Sorry.
OK, but that was an example. What are the real numbers?
Hey, I actually have some good news for you! Some tests are better than the example listed above, and it’s possible to find out which is which.
The U.S. FDA has a list of currently authorised tests that gives both their sensitivity and specificity. They even do the maths for you on both positive predictive value and its counterpart, negative predictive value. Here are a few examples:
This Cellex test picks up 93.8% of positives and 96% of negatives. Under the assumption that 5% of the population has had the virus, if you get a negative test result, there’s roughly a 99.7% chance that you are truly negative. But if you get a positive result, there’s only a 55.2% chance that you actually have the antibodies.
Not all tests have such a low positive predictive value, though. Here’s another:
This is a different type of test, and it’s more accurate. So accurate, in fact, that even if only 5% of the population has had the virus, the positive predictive value is still roughly 92.9%. So your positive result is more likely to be accurate than with the test we discussed in the example, but it’s still no guarantee. (Also notice that the confidence interval is pretty wide—92.9% is an estimate within a wider range.)
Regardless of which test you take, the recommendations still apply: Even if you got one of the more accurate tests, a positive result is no licence to go to a party and then cough on your grandma. For more on what antibody tests can and can’t tell you about your COVID-19 status, check out our post on exactly that.