Notes on testing in pandemic
Disclaimer: I am not an epidemiologist — I just read stuff on the Internet and use common sense and some basic math, and I am surprised that I cannot find similar analyses online.
I have started writing this as a kind of comprehensive framework for all of the testing cases that I discussed on the internet. It is kind of long.
Basics
We test so that we can make informed decisions on what actions to take. In the context of a pandemic the test tells us about the probability of the patient being infected (and infectious) or not and the main actions are: isolation (in many versions and degree of strictness), contact tracing and healing procedures. The decisions depend on the infection probability but also on the possible outcomes if the decision is wrong. If we quarantine someone who is not infected — how costly is that? If we don’t isolate someone who is infected — again what is the cost? More formally based on the probabilities we need to calculate the expected utility of the possible actions and choose the one with the highest outcome. The information about Covid19 we have is still very fragmented. We don’t know why the disease seemed to hit so strongly in some countries and regions and spare others. We know that it is very dangerous for old people and people with other diseases. But we don’t know much about its long term effects, there is some evidence that even in seemingly mild (and even otherwise asymptomatic) cases there might be severe long term consequences of Covid19. Our decisions have to be based on this imperfect information.
Computing the probabilities
There are many pieces of information that we can take into account when making the decisions. It is impossible to list everything that could influence the estimates, but the most basic one is background prevalence of the disease — how many cases we have in a given population. Then we have the tests outcomes, the symptoms, the contact networks etc. If the cost of a bad decision is big we should take into account as much information as feasible. It is not always obvious how to combine the various data points. If the information pieces are independent you can use the Bayes rule — but it is always a judgement how much independence there is.
The tests
There are two types of testing errors — false positives and false negatives. A false positive occurs when a test result is positive but the patient is not infected, and if there are few false positives we call the test sensitive. False negatives are when the test is negative but the patient is infected — if there are few false negatives the test is specific. There are many kinds of tests, here we mostly concentrate on PCR and antigen tests. PCR is the ‘gold standard’, it has both great sensitivity and specificity, but is costly and requires a lab. So far the public response to the pandemic concentrated on PCR tests, but we have no capacity to make enough of these tests. Antigen tests are less sensitive for detecting infections but are pretty good at detecting infectiousness, and have good specificity. They can also be very cheap, simple and not require any special equipment or help from a professional, in fact they could even be done at home. In practice, no matter of what test we use, we can never be 100% sure that a test result is right, because no test is perfect and people make mistakes, but also because the patient can become infected in the time between the taking of the sample and now.
You can use Bayes rule to calculate the probability that a patient is infected by combining the test sensitivity and specificity with the background prevalence of the disease in a given population — here is an online calculator for that: https://www.bmj.com/content/369/bmj.m1808/rapid-responses (the defaults for background prevalence are very high — they are probably adjusted for patients with some symptoms — but you can change all the input numbers).
The cases
Testing in hospitals
When a patient arrives in a hospital there is very low margin on being wrong. Because wrong treatment could cost his life — but also because he could infect other patients and staff if he is infectious and we don’t isolate him or get infected if he is not and we treat him together with other infected patients. That does not mean that we always need perfect tests, or even that we always have to test. For example we don’t test patients for common cold, even if the probability if any given hospital patient having a cold is not negligible. But the risk if we get it wrong is not that big compared to the cost of testing for all the possible common cold viruses. On the other hand we also don’t normally tests patients for Ebola — even though the consequences of a patient infecting staff or other patients with it would be probably be much worse then with Sars-CoV-2. The probability of the patient having Ebola is just too law. It would be different if the patient had symptoms of Ebola, or if he just returned from a region with Ebola outbreak or if both circumstances were true. The consequences of Sars-CoV-2 infection are not that severe as with Ebola, but much worse than common cold and we are in the middle of Sars-CoV-2 pandemic — so the probability of any given patient having it is not negligible and we probably want to test him with the best available tests and that means PCR.
It is not only other patients that can infect someone in the hospital —there are also visitors and staff. For now in the pandemic we just decided that we don’t let any visitors in — the risk is too high, and we cannot test everyone. Maybe if we had good rapid tests we could make a different decision. It is not costless, but it is reasonable for now. We have to let the staff in, so we need to test them for infectiousness and we need to do it often. We also cannot afford many false negatives — but if we cannot test them with PCR often enough — then antigen tests should be a good replacement. We could probably live with some false positives, the doctors would just get some rest.
Testing when visiting elders
It is not only hospital patients for whom the infection is dangerous — we know that it is very dangerous for older people, and people with some other medical conditions (for example taking immunosuppressive drugs). We need to evaluate the probability of their visitors being infectious and stop them if the probability is too high. That visitor is often ourselves and we have lots of very detailed information that can influence the estimate, facts like if we feel any symptoms of infection, if we met someone infected, if we live in areas with high prevalence, if we go into crowded places indoors, if we wear masks etc. If the estimated probability is too high, but the visit is very important we can test ourselves to be more sure. With strangers will never have the same level of background information — so the estimates will be much less precise and we might test them to be more confident. We probably don’t want to burden the public healthcare with our testing — so a at home antigen test would be great for this case.
Limiting the spreading of the virus
If we want to stop the pandemic and not just cope with its results, we need to isolate not only hospital patients and other fragile people — but all infectious carriers.
Symptomatics and contacts of confirmed cases
There are groups of people that have high probability of being infected even without the tests: people with symptoms and people with close and long time contact with a confirmed infectious carrier. If the symptoms are severe we send them to hospitals — this case is covered above. If the symptoms are mild the decision is between more or less strict home isolation and tracing their contacts to find more infected people (and isolate them). Even if the symptoms come from common cold— then home isolation is not a bad choice, because why should we spread other viruses? There are degrees of isolation, and strict isolation has quite high cost — so we might still want to do the test and adjust the isolation level. But if there is also other information suggesting that the symptoms are from Covid19 — for example the patient had also close contact with another confirmed case — then we have a very high probability that the patient is infected with Sars-CoV-2 and even a negative test result would probably not reduce that probability enough for us to change the decision to isolate. It might be better to not test the patient so that a negative test result does not give him a wrong impression and make him neglect the instructions to self-isolate. The situation is similar with close and long contacts of infectious people.
We can also go deeper and evaluate the symptoms, there are symptoms that give you higher probability of Covid19, like dry cough, and others that only slightly suggest it, like stuffed nose. And there are combinations of these, they are not entirely independent so the probability updates are not obvious — but at least the direction of it is — the more of the symptoms, the more severe — then the probability of Covid19 is bigger. And similarly we can also measure the length and closeness of contact.
Population wide testing (even asymptomatics)
If we do really mass testing we can expect many false positives. This will happen even with very good tests. With let’s say specificity of 99% — 1% of false positives out of a million is 10000. Fortunately we can test them again and, assuming they are somehow correlated, get let’s say 200 false positives. Not that bad, taking into account that 2 weeks of isolation is a low cost in comparison of letting the virus spread. But if the prevalence is very low (below 0.02% in our case)— than we could still get more false positives than true positives — such situation could undermine the testing procedures in the public view. So population wide testing is useful only for high prevalence. We can also select sub-populations with higher prevalence based on residence, profession and other data.
We can also test people with symptoms that have an important reason to break their home isolation. For example to visit a doctor for some unrelated issue.
In the example calculations above I assumed that the subsequent tests errors will not be completely uncorrelated. We can try to limit the correlations by using different tests or different procedures. Some false positives are not that bad as others — for example we can have two tests that falsely classify sample with a different virus as Sars-CoV-2 — but again the decision to isolate the patient infected with a different virus would not be that bad.
The testes suitable for mass testing are the cheap and simple ones. They don’t have to have perfect sensitivity, because we can still stop the disease even with not perfect tests if we do them frequently enough. But they need to have good specificity because false positives undermine public confidence in the tests. It is very important is that we need people to comply with the procedures and self-isolate if they get a positive result or if they have symptoms regardless of the test result.
Entry screening
We can test people at entrance to public gathering or visitors and staff of nursing homes. Or if inside the country we have low prevalence — then we could do entry screening on national borders. This is similar to the point above — but here we would select a subpopulation based not on the probability of infection — but on the high cost of error. If we let in someone infectious — he could infect many other people. So it might be useful even when the background prevalence is low.
The delay between the test and the results, makes the PCR tests here really inconvenient. Rapid antigen tests should fit this case much better.
At home tests and compliance
If we want to do really mass testing we cannot rely on the bottleneck of professional staff doing the tests — we need to let people do their own tests. Sure not everybody will do their tests and especially not everybody would comply with the costly isolation if they test positive. Fortunately the same calculations that show us that we don’t need a 100% sensitivity to squash the pandemic also show that something like 50% compliance is enough. I think this is realistic — but also I believe that we should not underestimate the ability of groups and networks of people to create and enforce rules among their members. Sure mass transit and the crowd at a shop might be a lost case — but most of our long and close contacts come from these stable networks. I am sure people will find ways to mutually negotiate rules in their families, workplaces, among friends etc.
My opinionated end notes
It is very hard to come with good guidelines — because in each individual situation we’ll have slightly different information available, and different consequences of our decisions. Each new piece of information could improve the outcomes and you never know when to stop your information gathering. It starts with the basics — there are countries like New Zealand that have practically no new cases inside the country, and there are countries with tens of thousands new cases — obviously these call for different measures. The circumstances can also change. But beside optimising the outcomes the guideline needs also to be simple enough that people understand it, and it also needs to be seen reasonable so that people follow it.
Test also play a role in contact tracing, and estimating the background prevalence of the disease. So there are yet more variables in our calculations.
The current organized response everywhere is a complete fiasco and chaos because institutions are not flexible enough. I am more and more convinced that before we try to have institutionalized procedures and rules we should let people organically develop them. We need to make sure that people have the right information about the tests, like their accuracy, and maybe some other very basic rules. But we should let people build the response organically in their smaller or bigger communities — like appartament buildings, schools, friend and family networks. And only then, when we see what works we should adopt it on the national scale. Testing is a case in point — I am waiting for at home, cheap, simple and rapid antigen Sars-CoV-2 tests.
