Part 5 — a test isn't "accurate" in the abstract. Its usefulness depends on who you're testing and what you already suspected.
Lessons 3 and 4 were about therapy questions — does this treatment work? This lesson shifts to a different kind of question, one you already met in Lesson 2's question-type table: does this test actually tell me what I think it tells me? A positive result on a bad test can do more harm than the disease it's chasing, and a negative result on a bad test can offer false reassurance. Getting comfortable with test accuracy statistics is how you tell the difference between a genuinely useful test and a confident-sounding number.
Every diagnostic accuracy statistic falls out of one table. You take a group of patients, run the test on all of them, and compare the test result against a gold standard — the best available way of knowing whether they truly have the disease (biopsy, culture, long-term follow-up, expert panel, etc.):
| Gold Standard | |||
| Disease Present | Disease Absent | ||
| Test Result | Positive | True Positivea | False Positiveb |
| Negative | False Negativec | True Negatived | |
| Total | a + c | b + d | |
Green cells (a, d) are where the test got it right. Rust cells (b, c) are the two kinds of wrong — and they carry very different consequences.
A false positive (b) means telling a healthy person they're sick — anxiety, further invasive testing, sometimes unnecessary treatment. A false negative (c) means telling a sick person they're fine — missed or delayed treatment. Which error matters more depends entirely on the disease and the test's role, and it's worth asking explicitly before you look at any numbers.
These two numbers describe the test itself, independent of how common the disease is in whoever you happen to test:
Sensitivity and specificity are properties of the test (and the population it was studied in), and in principle they don't change with disease prevalence — but what they let you conclude about an individual patient changes a great deal, which is exactly the trap the next section is about.
A patient with a positive result doesn't want to know "of people with the disease, what fraction test positive" — they want to know "given that I tested positive, what's the chance I actually have it?" That's a different question, answered by predictive values:
Here's the catch: PPV and NPV depend heavily on prevalence — how common the disease is in the population being tested — even though sensitivity and specificity don't. This is the single most common source of diagnostic-test confusion, and it deserves a worked example.
Take a test with 90% sensitivity and 90% specificity — a genuinely good test by most standards. Run it on 1,000 people twice, in two different settings.
Setting A — a screening population, 1% disease prevalence (10 truly have the disease, 990 don't):
PPV = 9 ÷ (9 + 99) = 9 ÷ 108 ≈ 8%. Of everyone who tests positive here, roughly 92% are false alarms — despite using a 90%/90% test.
Setting B — a specialist clinic, 50% disease prevalence (500 truly have the disease, 500 don't):
PPV = 450 ÷ (450 + 50) = 90%. Exact same test, exact same sensitivity and specificity — wildly different real-world meaning of a positive result.
Why this matters: this is why population-wide screening with a test validated in a high-prevalence specialist clinic can flood a low-prevalence population with false positives. The test didn't change. The context did.
Because PPV and NPV are prevalence-dependent, they don't transfer well between populations. Likelihood ratios (LRs) solve this: they're built from sensitivity and specificity, so they don't shift with prevalence, and they tell you how much a result should move your suspicion.
| LR+ value | Effect on suspicion |
|---|---|
| > 10 | Large, often conclusive increase |
| 5–10 | Moderate increase |
| 2–5 | Small increase |
| 1–2 | Rarely important |
| LR− value | Effect on suspicion |
|---|---|
| 0.5–1 | Rarely important |
| 0.2–0.5 | Small decrease |
| 0.1–0.2 | Moderate decrease |
| < 0.1 | Large, often conclusive decrease |
The formal way to use an LR is with pre-test odds × LR = post-test odds (via Bayes' theorem), which needs converting probability to odds and back. In practice, most clinicians use a shortcut — the Fagan nomogram — a printed or online chart that lets you draw a straight line from pre-test probability through the LR to read off post-test probability directly, no arithmetic required.
Sensitivity and specificity so far assumed the test gives a clean positive/negative answer. Many real tests actually output a continuous value — a lab number, a score — and someone chose a cutoff to call it positive or negative. Move that cutoff and both sensitivity and specificity change, in opposite directions.
Each point on the curve is a different cutoff. Move toward the top-left corner (high sensitivity, low false-positive rate) and you're at a stricter or looser cutoff depending on which way you moved — there's no free lunch: gaining sensitivity by loosening a cutoff always costs specificity, and vice versa. The diagonal line is what you'd get from a test no better than a coin flip. The area under the curve (AUC) summarizes overall discriminative ability in one number: 0.5 is useless, 1.0 is a perfect test, and most decent clinical tests sit somewhere around 0.7–0.9.
Where the cutoff actually gets set is a judgment call, not a statistical one — it depends on the relative cost of false positives versus false negatives discussed earlier. A cancer screening test might accept more false positives to avoid missing cases; a test that triggers major surgery might do the opposite.
The same three-question shape from Lesson 3 applies here, adapted to test accuracy: