Part 3 — you found the paper in Lesson 2. Now: can you trust it, what does it actually say, and does it apply to your patient?
In Lesson 2, you turned a clinical worry into a PICO question and found a paper that matches it — ideally a randomized controlled trial (RCT), since that's the strongest design for therapy questions. Having the paper is not the same as trusting it. This lesson walks through the classic three-question framework used to critically appraise an RCT.
Every major appraisal checklist (CASP, CEBM, JAMA Users' Guides) boils down to three questions, asked in this order:
Skipping straight to question 3 — "does this help my patient?" — without first checking validity is the single most common mistake. A big effect size from a badly designed trial is worth less than a modest effect from a rigorous one.
This is about ruling out bias. For an RCT, check these in order:
| Check | What you're looking for |
|---|---|
| Randomization | Was allocation to groups truly random (computer-generated sequence), not something predictable like alternating patients or odd/even birthdates? |
| Allocation concealment | Did the person enrolling patients know in advance which group the next patient would land in? If yes, selection bias can creep in. |
| Blinding | Were patients, clinicians, and outcome assessors unaware of group assignment? Single, double, or triple blind? |
| Similar groups at baseline | Was there a "Table 1" showing the intervention and control groups were comparable at the start (age, severity, comorbidities)? |
| Complete follow-up | Were dropouts few, and were they explained? A good rule of thumb: worry if losses exceed ~20%. |
| Intention-to-treat analysis | Were patients analyzed in the group they were originally assigned to, even if they didn't complete treatment? This preserves the value of randomization. |
| Equal treatment otherwise | Aside from the intervention being tested, were both groups treated the same (co-interventions, monitoring)? |
If a trial fails badly on several of these, question 2 and 3 barely matter — the numbers may not reflect reality.
Once you trust the method, extract the actual numbers. For a therapy trial with a binary outcome (e.g. "had a stroke" vs. "did not"), a few core statistics do most of the work:
Suppose a trial finds: 10% of patients on placebo have a stroke over 5 years, vs. 6% of patients on the new drug.
Why this matters: "40% relative risk reduction!" sounds dramatic in a headline. "You need to treat 25 people to prevent one stroke, and the other 24 get the drug's side effects with no benefit" is the same result, stated honestly. Always ask for the absolute numbers behind a relative-risk claim.
Also check the confidence interval around the main result. A 95% CI that's narrow and doesn't cross the "no difference" line (e.g. a relative risk CI that doesn't cross 1.0) suggests a precise, statistically solid finding. A wide CI that straddles "no effect" means the study may be underpowered — too small to be sure.
This is where the patient from Lesson 1 comes back into the picture. Ask:
This is also where your PICO question from Lesson 2 pays off: if you built it around your actual patient, this final step becomes a direct comparison rather than a guess.
Pro tip: the CASP RCT checklist and the CEBM RCT worksheet (linked in Lesson 1 and below) turn this into a fill-in-the-blanks form. Use one the first several times you appraise a trial — it becomes intuitive faster than you'd expect.