When Testing for Rare Disease Deceives
A patient worried about coeliac disease receives tissue transglutaminase (tTG) serology testing. The test is 95% sensitive (detects 95% of true coeliac cases) and 98% specific (correctly identifies 98% of non-coeliac people). Result: positive. Does the patient have coeliac disease?
Naive interpretation: 98% specificity suggests only 2% false positive rate. Conclusion: probably coeliac. But this reasoning ignores prior probability—the pre-test likelihood of disease in this population.
Bayes' theorem mathematically updates probabilities after testing: posterior probability = (sensitivity × prior probability) / [sensitivity × prior probability + (1-specificity) × (1-prior probability)]. This simple formula reveals why test interpretation hinges on prior probability.
In the general U.S. population, coeliac disease prevalence is 0.7% (prior probability = 0.007). Plugging values: posterior probability = (0.95 × 0.007) / [0.95 × 0.007 + 0.02 × 0.993] = 0.0067 / (0.0067 + 0.0199) = 25%. A positive test raises disease probability from 0.7% to 25%—substantial increase, but still 75% likelihood of false positive.
Conversely, testing in high-risk patients (positive family history, strong gastrointestinal symptoms) might have prior probability of 20%. Same positive test: posterior probability = (0.95 × 0.20) / [0.95 × 0.20 + 0.02 × 0.80] = 0.19 / (0.19 + 0.016) = 92%. Now the positive test is highly reassuring of disease presence.
Predictive values capture this more intuitively than sensitivity/specificity. Positive predictive value (PPV) is the probability of disease given a positive test; negative predictive value (NPV) is probability of no disease given negative test. These depend directly on prevalence.
For coeliac screening in the general population with 0.7% prevalence, a 95% sensitive and 98% specific test has PPV = 25% and NPV = 99.9%. For screening in a high-risk (20% prevalence) population, PPV = 92% and NPV = 99%.
Many diagnostic tests have low PPV in low-prevalence conditions. Lyme disease serology in non-endemic areas produces mostly false positives. Cardiac troponin elevation in critically ill non-cardiac patients typically reflects other causes. Positive tests for rare conditions in asymptomatic populations are frequently false.
Bayesian thinking transforms clinical practice. Before ordering a test, ask: what's the pre-test probability in this patient? If very low (rare disease, minimal symptoms), even a positive test may suggest only 10-30% disease likelihood. If very high (classic presentation, high-risk population), the test becomes confirmatory more than diagnostic.
Microbiome-disease associations illustrate this principle. Research associates Faecalibacterium deficiency with inflammatory bowel disease. But Faecalibacterium also changes in diet, antibiotics, and numerous other conditions. Using Faecalibacterium abundance alone to diagnose IBD in a low-prevalence population (general symptoms) produces mostly false diagnoses. In high-prevalence patients (chronic diarrhea, blood in stool, family history), the association becomes more useful.
Sequential testing improves diagnostic accuracy. Order first test in low-probability patients; if positive, follow with more specific confirmatory test. This approach reduces unnecessary follow-up and anxiety from false positives.
Understanding Bayesian thinking—that test results mean different things depending on prior probability—is fundamental to interpreting modern medicine intelligently.