Concept note · companion to the pipeline pair

The Closed Loop — Automating Hypothesis Generation Between Forward and Backward Clinical Pipelines

What happens when the backward pipeline's findings become the forward pipeline's next query, on repeat — and what the resulting "new knowledge" is actually worth.

Narrative summary Two pipelines already exist: a forward one that turns a plain-language question into a coded FHIR cohort, and a backward one that starts from an outcome and walks a cohort's timeline back to find what correlates with it. This note wires them into a loop — the backward pipeline's top finding becomes the forward pipeline's next query, run on a different dataset, whose outcome data feeds the backward pipeline again. Two human checkpoints keep this from running away: a reviewer approves each drafted query before it's run (section 03, section 08), and a reviewer validates each replicated result before it's treated as confirmed (section 07). Nothing in the loop invents a number — every lift, rate, and cohort count is a direct read off computed data (section 03); the LLM only ever proposes the next question and decides when to stop.

What comes out the other end is not a discovery and not a predictive model (section 06) — it's a replicated, human-approved correlational lead (section 05). But once a pattern clears both checkpoints, it doesn't just sit in a log: it re-enters as the next seed outcome, so the loop keeps building on its own prior confirmed findings (section 02, section 07). Framed plainly, that makes a validated loop pattern the same category of fact a textbook or clinical guideline already is — a confirmed clinical association — just reached by a different, machine-assisted route rather than trials or expert consensus, and logged with the same rigor (cohort, dataset, lift, reviewer) so it can be cited and built on the same way (section 07).

section 08 and section 10 make this concrete: section 08 is a fully live, in-browser demo of one loop round on an illustrative Pneumonia cohort, and section 10 goes further — it calls the real, public SMART Health IT R4 sandbox live, pulls up to 100 real patients for any of ten SNOMED CT-coded seed diseases, scans all three lanes (comorbidity, symptom, intervention) exhaustively, ranks the candidates by lift, and drafts a query card for each — every one awaiting a human's approve or reject, exactly as the loop's design requires.

01

Two one-directional pipelines, wired into a loop

The forward pipeline turns a question into a cohort. The backward pipeline turns an outcome into candidate questions. Run one after the other, on repeat, and the system stops needing a human to supply the next question at all — until it needs a human to check that the question is sensible.

Neither pipeline was built to run alone forever. The forward system resolves a clinician's phrase to SNOMED CT concepts and returns a coded cohort with analytics attached. The backward system starts from an outcome already sitting in that cohort — an escalation, a readmission — and walks the coded timeline backward to find what recurs more than baseline. Put the backward system's output back into the forward system's input slot, and each round of the loop replaces a step a human used to do by hand: deciding what to ask next.

The loop in one line backward finding → phrased as a query → forward pipeline runs it on a different dataset → new outcome data → backward pipeline runs again on that → next finding.
02

Wiring: automated stages vs. the two checkpoints

Everything below the two red bars runs without a person present. The red bars are where a person reads what the loop produced and decides whether it's allowed to continue.

Dataset A (original) Backward pass + pattern analysis ranks findings by lift Query verification human checkpoint approve / reject Forward pass + outcome analytics on Dataset B Dataset B (replication) Outcome validation human checkpoint confirm / discard if validated: confirmed pattern re-enters as the next seed outcome
automated — backward pipeline automated — forward pipeline human checkpoint loop-back path
B
automatedBackward pass on Dataset A — replay coded timelines back from a seed outcome, pool into frequency tables, compute lift vs. baseline.
B
automatedPattern analysis — rank recurring items by lift × cohort size; attach the exact query that reproduces each one.
B
automatedPattern insight — orchestrating LLM reads the ranked table, drops findings below a lift/size threshold, and drafts the next natural-language query from the top surviving pattern.
H
human checkpointQuery verification — a reviewer reads the drafted query, confirms it's a sensible thing to ask (not a leading question, not underpowered, not already answered), and approves or edits it before it runs anywhere.
F
automatedForward pass on Dataset B (a different sandbox, site, or time window) — deterministic phrase→concept lookup resolves the approved query to SNOMED tokens, runs it as a live FHIR search, returns a coded cohort.
F
automatedOutcome analytics — cohort analytics layer computes match rate, hub nodes, and the same outcome measured on the new dataset.
Match rate— no data
Hub nodes— no data
Outcome— no data
Illustrative only — this concept note doesn't wire up the full analytics layer; see the live demo below for the one outcome metric (escalation lift) that is actually computed.
H
human checkpointOutcome validation — a reviewer compares the replicated result to the original finding: did the pattern hold, at what lift, in a population it wasn't discovered in? This is the only place the loop's output becomes a "validated" lead rather than a candidate.
↻ if validated, the confirmed pattern re-enters as a new seed outcome for the next backward pass
Loop repeats · each round replicates on a further dataset or narrows the cohort · orchestrating LLM only ever proposes the next query, never the finding itself
03

What the orchestrating LLM is actually allowed to touch

Both source pipelines share one non-negotiable design rule: generative AI never invents a cohort statistic, a hub node, or an outcome — those are always direct reads from computed data. The loop inherits that rule exactly. The LLM's job is entirely in the seams between the two deterministic engines, not inside either one.

LLM does

Reads the ranked lift table. Chooses which finding clears the threshold to act on. Phrases that finding as the next plain-language query. Decides when to stop iterating (diminishing lift, budget of rounds, or a finding a human should see now).

LLM never does

Computes a lift ratio. Decides a cohort matches a concept. Writes an outcome rate. Fabricates a comorbidity link. Every number in the loop still comes from the phrase→concept lookup table and the analytics layer — never from a model's completion.

04

One worked round

StepOutput
BBackward pass on the Pneumonia cohort, seed outcome = ICU escalation. Top finding: chronic kidney disease present → 2.4× escalation lift, n=18.
BLLM drafts next query: "pneumonia patients with chronic kidney disease"
HReviewer approves — cohort size looks adequate, the comorbidity is clinically plausible, not already a known guideline trigger.
FForward pipeline resolves the phrase deterministically, runs it against a second FHIR endpoint (a different sandbox slice), returns 22 matched patients.
FAnalytics on the new cohort: escalation rate 58% — close to the original 61%.
HReviewer validates: pattern replicated within a reasonable band on an independent pull. Logged as a confirmed lead, not a clinical rule.
05

So what is the new scientific knowledge worth?

Automating everything except the two human checkpoints changes throughput, not the epistemic category of what comes out the other end. Two full loop passes on independent datasets produce a replicated correlational lead — that is a real and useful thing, but it is a specific rung on the evidence ladder, not a discovery.

What's actually new

Not the medical fact itself — CKD raising escalation risk is not a surprise to clinicians. What's new is that the pattern was found without a human deciding to look for it, is fully reproducible (the exact query is attached to every finding), and has already been checked against a second, independent dataset before any person spent real time on it.

Where the value actually sits

In collapsing months of manual chart-review and hypothesis-drafting into two review checkpoints. The humans spend their attention curating from a pre-filtered, pre-replicated candidate pool instead of generating candidates from a blank page. That's a throughput and coverage gain, not a truth gain.

The honest claim The loop's output is worth exactly what "a well-formed, cross-dataset-replicated, human-approved lead" is worth. That is real scientific value — it is the input to a study's hypothesis section — but it stops there.
06

What no amount of looping produces

07

What a validated pattern actually becomes: a second path to the same kind of ground truth

The loop-back arrow in the diagram above (section 02) is doing more work than it looks like. Once outcome validation (human checkpoint 2) passes, the confirmed pattern doesn't just get filed away as a one-off result — it re-enters the system as the next seed outcome for another backward pass. That's the mechanism worth naming plainly.

Textbooks and guidelines: ground truth built one way

Clinical guidelines encode "known associations" — CKD raises escalation risk in pneumonia, hypoxia predicts deterioration, and so on — arrived at through trials, registries, expert consensus, and years of accumulated chart review. That corpus is the ground truth clinicians already carry into every encounter.

A validated loop pattern: the same kind of fact, a different route there

A pattern that has cleared both checkpoints — drafted from a real backward scan, replicated on an independent dataset, and signed off by a reviewer — is the same category of fact: a machine-surfaced, human-confirmed clinical association. It didn't come from a trial or a guideline committee, but it was found, tested against new data, and checked by a person before being logged. That is a second, mechanized path to the same destination — not a shortcut around it.

This is exactly why the feedback step matters: every validated pattern is logged and mapped the same way a guideline citation would be — with its cohort, its lift, the dataset it replicated on, and the reviewer who signed off — so it can be treated as a durable reference point, not just a transient chat answer. Once logged, it can legitimately seed the next backward pass, the same way an existing guideline fact would prompt a clinician's next question. Round over round, this is how the loop accumulates its own small, audit-trailed body of machine-generated clinical patterns — sitting alongside textbook knowledge, not replacing it.

Same evidence tier, not a shortcut A validated loop pattern still sits at the "replicated correlational lead" rung described in section 05 — it is not a randomized trial and it does not outrank a guideline that contradicts it. What changes is that it is now logged as ground truth for this pipeline's own purposes: a stable, citable fact the next backward pass — human or machine-initiated — can build on, exactly the way a clinician builds on a guideline they already trust.
08

Live demo — run one round of the loop

This runs live in your browser, scanning all three lanes the backward pipeline tags — comorbidity, symptom, and intervention — across 8 candidate factors on the same Pneumonia cohort, not just the 3 comorbidities used in the worked round above. Every ranked finding, drafted query, and lift figure is computed by the script on this page as you click — nothing is precomputed.

What's real vs. illustrative The patient-level counts are synthetic, built to match the CKD/T2D/COPD figures used earlier plus five additional candidate factors (obesity, asthma, three symptom/intervention flags) drawn from the same illustrative Pneumonia cohort. The ranking math, query drafting, forward replication, and pass/fail validation are the genuine computation the loop describes — not staged numbers. One factor (asthma) is included deliberately as a negative result, so the demo shows what "doesn't replicate" looks like too, not just hits.
1

Backward pass on Dataset A automated

Seed outcome: ICU escalation. Rank all 8 candidate factors — across comorbidity, symptom, and intervention lanes — by escalation lift over the no-flag baseline.

2

Query verification human checkpoint

Edit the drafted query if you want, then approve or reject it before it reaches Dataset B.

3

Forward pass on Dataset B automated

Runs the approved query against a second, independent cohort — the replication test.

Match rate— no data
Hub nodes— no data
Outcome— no data
Match rate and hub nodes aren't computed in this demo. Escalation lift below is the one outcome measure that's live.
4

Outcome validation human checkpoint

Compare Dataset A's finding to Dataset B's replication and decide whether it clears the bar for a confirmed lead.

09

Guardrails carried over from both source pipelines

Human-in-the-loop by design Every finding the loop produces is a candidate question for a reviewer, never a finding a committee should act on unread. Automating the rounds between the two checkpoints is meant to give humans more, better-triaged rounds to review — not fewer humans in the process.

Practically, that means: a query-verification step that can reject a drafted query outright (underpowered, leading, already settled), a hard floor on minimum cohort size before any lift number is allowed to surface, and a log of every accept/reject decision so the templates that never survive review get pruned rather than repeated indefinitely.

10

Live backward query generator — real SMART Health IT sandbox data

Everything above this line is illustrative. This section is not: it calls the public SMART Health IT R4 sandbox live, in your browser, for whichever seed disease you pick below (the same ten concepts used in the forward pipeline's demo). In one run it pulls up to 100 matching patients from a single Condition search, then walks each patient's full record — Condition, Observation, Procedure, and MedicationRequest — and drafts every qualifying backward candidate across all three lanes the forward pipeline tags: comorbidity, symptom, and intervention. Nothing is auto-run against the data further: every card below is a draft for a human to verify or reject, with the exact query and lane shown next to the numbers that produced it.

Shared public sandbox This is the same free, shared SMART Health IT test server the forward pipeline uses — synthetic (Synthea-style) patients, not real people, and the pool can be small, uneven, or change over time. Treat every card as a hypothesis, not a result.
1

Pick a seed disease

Same ten SNOMED CT concepts tagged in the forward pipeline's demo.