Evidence-Based Medicine · Course for Techies

Lesson 6: Prognosis & Harm Studies

Part 6 — you can't randomize someone into smoking, or into a rare side effect. This is where the RCT stops being the right tool.

Lessons 3 and 4 covered therapy questions, where the RCT reigns because you can randomly assign an intervention. Lesson 5 covered diagnosis. This lesson covers two more question types from Lesson 2's table — prognosis ("what's likely to happen to this patient over time?") and harm ("does this exposure cause that outcome?") — both of which are usually answered with observational designs, not trials.

Why this matters for techies: think of this as the difference between an A/B test and an observational log analysis. An RCT is the A/B test — you control the assignment, so confounding is handled by design. A cohort or case-control study is more like mining production logs for correlations between a feature flag and an outcome nobody explicitly randomized — powerful, sometimes the only option, but you now have to work much harder to rule out that some third variable is driving both.

Why Not Just Run an RCT?

Three practical and ethical walls push prognosis and harm questions out of RCT territory:

You can't ethically randomize harm. You cannot assign one group of people to smoke, take a teratogenic drug, or be exposed to a pollutant to see what happens. If you already suspect harm, deliberately causing it is off the table.
Rare outcomes need huge samples. If a side effect happens in 1 in 10,000 patients, an RCT with a few hundred participants will see zero events in either arm — not because the drug is safe, but because the trial was never powered to find out.
Long-term outcomes need long follow-up. Prognosis and many harms unfold over years or decades. Trials are expensive to run and expensive to keep patients enrolled in for that long, so most stop long before the outcome that matters most has had time to occur.

Observational studies trade the clean confounding-control of randomization for the ability to study questions an RCT simply cannot ethically or practically answer.

Cohort Studies

A cohort study follows groups defined by exposure status forward in time and compares how often the outcome occurs in each group.

Cohort Study — Direction of Travel

Both groups start free of the outcome. You wait, and count how many in each group go on to develop it. This is a natural fit for questions like "do smokers develop lung cancer more often than non-smokers?" or "do patients on this drug have more kidney injury than those on an alternative?"

Type	How it works	Trade-off
Prospective cohort	Enroll exposed/unexposed groups now, follow them forward, watch the outcome occur	Best data quality and timing, but slow and expensive — can take years
Retrospective cohort	Use existing records to reconstruct exposure and outcome that already happened	Fast and cheap, but data quality depends entirely on how good the historical records were

Case-Control Studies

A case-control study runs in the opposite direction — it starts from the outcome and looks backward for exposure.

Case-Control Study — Direction of Travel

You identify people who already have the outcome (cases) and similar people who don't (controls), then look back at their histories to compare how often each group was exposed. This is the practical choice when the outcome is rare or slow to develop — you'd otherwise need to follow an enormous cohort for decades to accumulate enough cases. A study of a rare childhood cancer, for instance, is far more feasible as a case-control design than as a cohort.

Rule of thumb: rare outcome → case-control (you actively go find the cases). Rare exposure, or you need to know the actual rate of the outcome — not just compare it — → cohort. Case-control studies can estimate an odds ratio as an approximation of relative risk, but cannot directly report incidence, since the proportion of cases and controls was set by the researcher, not by nature.

Hazard Ratios and Survival Data

Prognosis questions usually care not just about whether an outcome happens but when. A drug that delays death by five years and one that delays it by five days could show the same eventual event rate but very different value to a patient. This is where survival analysis and the hazard ratio (HR) come in.

Hazard ratio compares the instantaneous rate at which an event occurs in one group versus another, across the whole follow-up period — not just the total count at the end.

HR = 1 → no difference in event rate between groups
HR < 1 → exposure/treatment group has a lower event rate (protective)
HR > 1 → exposure/treatment group has a higher event rate (harmful)

You'll usually meet hazard ratios alongside a Kaplan-Meier curve — a step-down plot of the proportion of patients still event-free over time, one line per group. Two curves can end at the same final percentage but separate very differently along the way; a curve that splits early and stays apart tells a different clinical story than one that only diverges near the end. As with any effect estimate, check the confidence interval around the HR — the same logic from Lesson 3 applies: a CI crossing 1.0 means the result isn't statistically distinguishable from no effect.

Confounding: The Central Threat to Observational Studies

Because exposure wasn't randomly assigned, people who got the exposure may differ systematically from those who didn't — in ways that also affect the outcome. That third variable is a confounder, and it's the main reason observational associations are held to a lower standard of proof than RCT results.

The Confounding Triangle

A confounder is linked to both the exposure and the outcome, and can create or hide an association between them even if the exposure has no real causal effect at all.

Classic Example

Observational data might show that people who carry lighters have higher rates of lung cancer. Carrying a lighter doesn't cause cancer — smoking is the confounder: it drives both lighter-carrying and cancer risk, creating a spurious association between two things with no direct causal link.

How Studies Try to Handle It

Matching — pairing exposed and unexposed (or cases and controls) on known confounders like age and sex before comparing them.
Stratification — analyzing the association separately within subgroups defined by a suspected confounder.
Multivariable adjustment — statistically controlling for measured confounders in the analysis (the "adjusted" in "adjusted hazard ratio" or "adjusted odds ratio").
Restriction — only enrolling a population narrow enough that the confounder can't vary, e.g. studying only non-smokers.

All four only work for confounders the researchers thought to measure. Residual confounding — from a confounder that wasn't measured, or wasn't measured well — can never be fully ruled out in an observational study. This is exactly why RCTs remain the gold standard where they're feasible: randomization balances every confounder, known and unknown, purely by chance, given a large enough sample.

Bradford Hill: When Is an Association Probably Causal?

Since observational studies can't prove causation the way a well-run RCT can, epidemiologists lean on a set of considerations — originally proposed by Sir Austin Bradford Hill — that make a causal explanation more or less plausible. None is individually decisive, but together they're a useful checklist:

Consideration	What it asks
Strength	Is the association large, or barely above noise?
Consistency	Has it been seen repeatedly, in different populations and studies?
Temporality	Did the exposure clearly come before the outcome? (The one truly essential criterion.)
Dose-response	Does more exposure track with more outcome?
Biological plausibility	Is there a credible mechanism connecting exposure to outcome?
Reversibility	Does removing the exposure reduce the risk?

Appraising a Prognosis or Harm Study

The now-familiar three-question shape, adapted once more:

Are the results valid? Were exposed/unexposed (or case/control) groups similar apart from the exposure? Were outcomes measured the same way, and blindly, in both groups? Were the major plausible confounders identified and adjusted for? Was follow-up long enough and complete enough to catch the outcome?
What are the results? What's the relative risk, odds ratio, or hazard ratio — adjusted or unadjusted — and its confidence interval? For prognosis specifically: what's the absolute risk over a clinically meaningful time frame, not just a relative comparison?
Will this help my patient? Does your patient resemble the study population closely enough that the confounding-adjustment holds? Is the exposure or risk factor something modifiable, or purely informational?

Extending Lesson 3's code-review analogy: a confounder is like an untested interaction effect between two features that your A/B test would have silently balanced out through randomization, but that an observational log analysis has to go looking for explicitly — and might miss entirely if nobody thought to check.

Homework for Lesson 6

Find a cohort or case-control study relevant to a prognosis or harm question for your patient from Lesson 1. Identify which design it is and why that design fits the question (rare outcome? long follow-up needed?).
List the confounders the study explicitly adjusted for. Name at least one plausible confounder you think might have been missed.
If the study reports a hazard ratio or Kaplan-Meier curve, describe in one sentence when the two groups' curves separate and what that timing implies clinically.
Run the association through as many Bradford Hill considerations as the paper lets you evaluate. Does it look more or less likely to be causal?

Resources for This Lesson

CASP UK — Cohort Study Checklist CEBM Oxford — Critical Appraisal Tools CEBM — Prognosis and harm calculators PubMed — search medical research papers