Screening Tests, Part Two

Last time we talked about screening tests, we talked about the definition of concepts such as sensitivity, specificity, positive predictive value, and negative predictive value. I told you at the end about an interesting effect observed in the relationship between positive predictive value (PPV) and prevalence. The effect is that as prevalence rises, so does the PPV. However, as prevalence rises, the negative predictive value (NPV) comes down. Let's analyze that and look at some examples... Ready for it? After the jump...

Hilarity ensues

A QUICK STORY
There once was a woman I worked with who had a relative with hepatitis C. She had done a lot of reading about hep C, and she became somewhat of an advocate and activist about the disease. One of her "projects" was to lobby the hospital where we worked to sponsor a screening of the entire town's population for hep C. She had posters, numbers, and a very compelling personal story. Unfortunately, she didn't have epidemiology on her side. Guess who did?

I DID
The town where we worked had an incredibly low prevalence of hepatitis C. It is a rural town where drug use and abuse, although present, is not exactly rampant. The test that my coworker was pushing is a screening test with about 95% sensitivity and 80% specificity. Remember that the test will call positive only 90% of the people who truly have hep C, and it will call negative only 80% of the people who don't. With that in mind, let's look at what different prevalence levels would mean in this scenario.

REMEMBER THE GRAPH
Remember how I showed you a graph last week that pretty much spelled out the relationship between PPV, NPV, and prevalence:

Remember?

You can very much just stop right here and understand that the relationship exists and just take it as it is. But you wouldn't be here if you just took stuff on "faith," would you?

Remember this set-up for the examples below...

SOME EXAMPLES
Staying with the hep C theme, let's look at different scenarios of a screening test for hep C with 95% sensitivity, 80% specificity, and a population of 100,000, with a varying prevalence from 50% (half of the people are infected) down to 0.1% (1 in 1,000 people are infected). Pay attention to the PPV and NPV... And to the sheer number of false positives that would be scared to death to be told they had hep C!

In this example, the prevalence is 50%. That means that 50% of the population, or 50,000 people, truly have hepatitis C. Our screening test has a sensitivity of 95%, meaning that it will identify 47,000 of those 50,000 people. That means that 2,500 people will not know they have hepatitis C. What about the other half without hep C? Well, because of the test's 80% specificity, 10,000 people without hep C will test positive. That's right... 2,500 people with hep C won't know, while 10,000 without it will think they have it. Notice, however, that the PPV is 82.6%, meaning that the test will be correct with regards to a positive test about 83% of the time. A physician looking at a positive test would be that confident that the test is correct. Notice that the NPV is 94.1%, meaning that the test will be correct with regards to a negative test about 94% of the time.

Let's look at what happens when prevalence is only 30%.

With prevalence at 30%, the PPV is lower than our previous example. The NPV, however, is higher. This is because of simple math. Remember that numerator of PPV is A (28,500), a number smaller than the numerator in the previous example (47,000) because the number of sick people is lower. That has an effect on PPV. Less sick people means more healthy people. More healthy people means that the numerator for the NPV (D or 56,000) is larger now, so NPV will be larger, too.

And that's the gist of it. Less sick people mean more healthy people. Less sick people mean smaller PPVs and bigger NPVs. Just watch...

 

See how PPV got ridiculously low? If a physician were testing someone at random in a population where the prevalence of the disease is 0.1%, would the physician trust a positive result? God, I hope not. (Yes, there's a reason why I wrote "someone at random" in bold. You'll see in a bit.)

What about the "near-perfect" test? One with 99% sensitivity and 99% specificity? It should be darn near perfect in picking up all the cases, right? Nope. The same concept applies. Observe:

See how the PPV drops while the NPV increases? It's the same effect, only not as pronounced because, by it's nature of 99% sensitivity and specificity, the test is picking up a lot more people who are sick and properly identifying as negative a lot more people who are healthy. But what if prevalence is 0.1%? Watch:

PPV becomes useless again. Apply this to the 2.5% estimated prevalence of hepatitis C in the US, and you get the following:

If we took 100,000 random people in the United States, we would miss 125 true cases while telling another 975 that they had tested positive and needed follow-up. Of course, this is assuming we use a near-perfect screening test. From a Public Health point of view, we could rule-out the negatives, but the positives would give us headaches.

DISCUSSION
When I discussed this concept to my coworker, she was incensed. (And I don't mean she was covered in aromatic spices and lit on fire.) She asked me how I could be so "heartless" so as to not want to identify people with hep C and offer them guidance and help. She was even more angry when I presented these figures to the committee organizing that year's health fair, the ones who were going to decide whether or not to offer a hepatitis C screening to anyone and everyone randomly going to the health fair. (There I go writing "random" in bold again.) The committee decided against screening participants for hepatitis C and went with the old standby of a lipid panel and a blood glucose test for people who were fasting.

IT MADE SENSE
Why did it make sense to screen for heart disease and diabetes? Because the prevalence of those two diseases were higher in that town that hepatitis C and because we were not using binary (positive/negative) screening tests with the limitations thereof. That pretty much guaranteed that those with high lipid counts and/or high fasting blood glucose levels indeed had a metabolic issue. Furthermore, it prevented a lot of people from being told, "Hey, your screening test was positive, so you need to follow-up." All that when they more than likely didn't have hep C because the PPV was less than 50% due to the low prevalence of hepatitis C in that town. (The prevalence was closer to 0.01% in that town. That's why I kept using that number in the examples.)

ALRIGHT, WHAT'S WITH THE BOLD RANDOMNESS?
I mentioned that these kinds of observations were true if random people were tested. If you randomly test a population, you'll have to deal with the effects I've described between prevalence and the number of people you scare with a false-positive result. Hopefully, health care providers and other people doing tests are not just picking people off the street and using something that is less than a gold standard.

Have you seen a recent HIV/AIDS screening test campaign? They don't exactly invite everyone and anyone to get tested, do they? No. They ask people with risk factors for having contracted HIV to come in and get tested. Why? Because that increases the prevalence of the people you are testing. That is, it is more likely for the screening test to truly identify positives. That's what it's all about in Public Health interventions. You could care less about the healthy people... For the purposes of infectious disease investigations and interventions, of course. You wouldn't go testing a 79 year-old nun from a convent in rural Italy for HIV, would you?

Probably not

And that's also why health care providers will want to know your signs and symptoms as well as our medical history before ordering a screening test like a strep throat test, a flu test, or even an infectious mono test. Knowing that increases the prevalence in the population (sick patients) that are to be tested, improving the PPV of the screening test overall.

POP QUIZ
Head over to PLos ONE and check out this paper on rapid influenza testing. Note the differences in sensitivity and specificity based on the time of onset of the patients. Now, answer me this... Why is there a difference in the sensitivity and specificity based on the time of onset?

AN OLIVE BRANCH
I offered my then coworker to work with her to develop a scheme whereby we could ask people about certain risk factors for hepatitis C and then screen only those who met certain criteria. Sadly, she was too mad at me for derailing her plan to go along with me on a new plan. From my point of view, it didn't take a rocket scientist to see that testing thousands of people in that town for hepatitis C would have been very, very expensive, and it would have caused far more problems than addressing the issue. (Which was lucky, because we didn't have any rocket scientists around.) It did, however, take a budding young epidemiologist.

One that is thankful for your attention and time.

Powered by ScribeFire.