Oral swab HIV test produces false positives

There's a new HIV test, which uses an oral swab—much safer and more convenient for both the patient and medical personnel—to collect the sample. Unfortunately, about a quarter of those in San Francisco who have tested positive on it turn out not to be HIV positive after all.

There is speculation that perhaps there's something special about the group tested, as this problem hasn't been seen before with this test. But there's also something else at work here, a basic misunderstanding of detection systems, and of Bayes' theorem.

Detection systems can fail in two basic ways, known as Type I and Type II errors. Type II errors are what people normally think of when they think of an alarm failure—the burglar breaks in, but the alarm doesn't go off. In medical tests, this is called a false negative, which means that the patient had the condition but the test didn't detect it.

The false positives in this case are also known as Type I errors, which are also a problem. For one thing, too many Type I errors can cause people to disregard the result, as when we all ignore car alarms. In medicine, a false positive can cause needless worry, Just ask any woman who's been told that there are "abnormalities" on her Pap smear, a notoriously error-prone test.

Detection system can be tuned to have very low Type I error rates, or very low Type II error rates, but usually not both at once. In public medicine, it usually makes sense to have a cheap safe screening test with a very low false negative rate, and then a more expensive but more sensitive follow-up test for those who test positive. It sounds like this new test would be ideal for an initial HIV screening test.

The other thing that many people don't understand about testing systems is Bayes' theorem. Consider an HIV test that's "99% accurate". That is, when someone is known to be HIV+, this test will detect it with 99% probability; similarly if they're HIV-, the test will show that 99% of the time. The problem is, that's not how the test is used. Such a test is done on a person whose HIV status is unknown, and the test comes back positive. The question is, what's the probability that the person really is HIV+?

Bayes' theorem tells us that the answer depends on the incidence of HIV in the population. Let's run the numbers on a population of 1 million Americans. The prevalence of HIV/AIDS in the US is about 0.33%. That means that of that 1 million, 3300 will be HIV+, and 996700 will be HIV-.

Of the 3300 HIV+ people, 3267 (99%) will test positive, and 33 (1%) will test negative.

Of the 996700 HIV- people, 986733 (99%) will test negative, and 9967 (1%) will test positive.

Now comes our poor testee, whose test comes up positive. Remember we don't know if he's one of the 3267 HIV+ people who would test positive, or one of the 9967 HIV- people whose tests produce false positives. So the chance that he's really HIV+ is 3267/(3267+9967), or 24.7%. In other words, of the people who test positive, only about 1 in 4 is really HIV+. And that's with a test with 99% accuracy!

Given all that, I think that this new test sounds pretty good. They just need to educate people that a positive result just means that further testing is needed, and not that they necessarily really are HIV+.

Technorati Tags: AIDS