Clearing up the Covid-19 Test Accuracy Debate with Data Science

Last October, my entire household came down with a nasty cold and cough. I was laid out flat on the couch for a couple of days and went for a Covid-19 test on day 4. Thankfully the test came back negative and I breathed a sigh of relief. As soon as I felt better I went on my merry way, confident that the test was highly accurate. I was wrong. 

Is Your Negative Covid-19 Result  Accurate?

So, how accurate is a Covid-19 test? According to the FDA [1], “No test is 100% accurate all of the time”, leading one to (incorrectly) believe that the test is close enough to 100%–perhaps 98 or 99%. The fact is–you can’t trust a negative result at all. According to Robert H. Shmerling, MD, Senior Faculty Editor for Harvard Health Publishing [2] “the true accuracy of tests for COVID-19 is uncertain.” There are many reasons for this, including the newness of the virus, a lack of reported accuracy data, and timing of the test in relation to symptoms. 

There are stories abound of people who tested negative only to find out they actually should have tested positive. One 34-year-old man with Covid-19 who tested negative four times before finally testing positive. Many experts suggest that anyone with symptoms should assume they are infected [3]. This makes one wonder about the usefulness of getting a test at all: why bother with the expense and time if you should self-quarantine anyway?  

Calculating a Probability for “Uncertain” Results 

As a statistician, I find it difficult to accept “we don’t know” as an answer. Given enough data, one should be able to ascertain a probability for anything–even if the result is a confidence interval [no term] wider than the Grand Canyon. Given that there were over 100,000 published articles on the coronavirus in 2020, one would think that there was enough data to calculate a few probabilities.  A recent tool published by Jara et al. did just that, creating a user-friendly tool that predicts the probability your negative result was actually correct. The tool is based heavily on the work of Kucirka et al. [5], which estimated the false-negative rate by day since infection. The tool creators then adjusted a hierarchical logistic model to estimate the false negative rates based on days since the onset of symptoms (hierarchical models group data into distinct levels–in this case, days since onset). Then the authors generated a Markov chain of 420,000 samples, discarding the first 20,000 and resampling [no term] the remainder to generate a 20,000 point sub-chain.

What was the Probability my Result Was Truly Negative?

I used the tool to find the probability that my October Covid-19 negative test result was accurate.

To use the tool, select one, two or three tests from the top tabs. For me, that was one test. Next, plug in your estimated probability of having Covid-19. I had all of the symptoms, so I estimated it was 50/50 (either Covid or a common cold). Next, set the slider at the number of days from first symptoms until the test. For me, that was 4. Here is the graphic for my results:

According to the chart, I had a roughly 20% chance of having Covid-19, despite having a negative test.

This tool makes it easier to have confidence in what your Covid-19 test results mean in terms of probability. This is a fascinating use of data to create a very understandable tool. This is what data science is all about–taking uncertainty and confusion and turning it into a very user-friendly tool that can be used and understood by everyone. 

References

[1] Coronavirus Disease 2019 Testing Basics

[2] Which test is best for COVID-19?

[3] How accurate are the laboratory tests used to diagnose COVID-19?

[4] Alejandro Jara, Eduardo A. Undurraga, Rafael Araos (Preprint, 2021).Tool for estimating the probability of having COVID-19 with one or more negative RT-PCR results. Tool available here.

[5] 9. Kucirka LM, Lauer SA, Laeyendecker O, Boon D, Lessler J. Variation in false-negative rate of reverse
transcriptase polymerase chain reaction–based SARS-CoV-2 tests by time since exposure. Ann Intern Med.
2020.

Covid19 test kit image: Secretaria Especial de Saúde Indígena (Sesai), CC BY-SA 2.0 a href=”https://creativecommons.org/licenses/by-sa/2.0%3E”>https://creativecommons.org/licenses/by-sa/2.0>;, via Wikimedia Commons