LSMA

LONDON SCHOOL OF MULTIOMICS & AI

Free course

Free course

0 of 11 lessons

Lesson 2 of 2

Module 2 · Judging a model

Evidence: from validation to trials

Evidence: from validation to trials

Evidence: from validation to trials

A model's numbers are only as strong as the study that produced them. This lesson sets out the evidence ladder for medical AI, and the reporting standards that tell you what a good study looks like.

⏱ 12 min readModule 2 · Judging a modelFree
Lesson at a glance3 steps · about 12 min
  1. The evidence ladder
  2. The rungs in plain words
  3. Reporting standards

The evidence ladder

Each rung of the ladder below answers a harder question than the one beneath it. The lower rungs ask "does the model get the right answer?"; the upper rungs ask "are patients better off when staff use it?". Rung 1, listed first, is the weakest evidence; rung 5, listed last, is the strongest.

In Measuring performance you learned to read sensitivity, predictive values, AUC and calibration. Those figures can come from any rung of this ladder, and the same number means much more when it comes from a higher one. Climbing the ladder is slow and costly, which is why most tools stop near the bottom.

1 · Internal validationTested on held-out data from the same source. Useful, but optimistic.Weakest
2 · External validationTested on data from other hospitals, ideally other countries and equipment.Stronger
3 · Silent prospectiveRuns live on new patients, but its output is hidden from staff, so its performance can be measured safely.Real-world
4 · Clinical studyStaff use the output, and patient outcomes are measured.Impact
5 · Randomised trialPatients or sites randomly use the tool or not, so its effect on outcomes can be isolated.Strongest

The rungs in plain words

  • Internal validation. "Held-out" data is a part of the original dataset that was set aside and never used for training, the test set from Module 1. It is optimistic because it shares the hospital, the equipment and the recording habits of the training data.
  • External validation. The unchanged model is tested on data from elsewhere. Both rungs 1 and 2 are usually retrospective: they use records that already exist, collected before the study began.
  • Silent prospective. Prospective means the study follows new patients forwards in time, as they arrive. Running silently shows how the model copes with live data, which is often messier and later than the cleaned records used in development.
  • Clinical study. Now staff see and act on the output. This is the first rung at which the tool can change care, for better or worse, so outcomes such as time to treatment or length of stay can be measured.
  • Randomised trial. Chance, not choice, decides who gets the tool, so the two groups are alike in everything except the tool. When whole wards or hospitals are randomised instead of individual patients, the design is called a cluster randomised trial; it suits tools that change how a whole team works.

Why is randomisation worth the trouble? The obvious alternative is a before-and-after study: compare outcomes in the year before a tool was introduced with the year after. The problem is that many things change in a year. Staffing, guidelines, the mix of patients and the season all shift, and a hospital that installs a new tool is often making other improvements at the same time. Staff who know they are being watched may also work differently for a while, whatever tool they are given. A before-and-after study cannot separate the tool from all of this; a randomised comparison running at the same time can.

Randomising individual patients is often impractical for AI, because staff cannot easily use a tool for one patient and ignore it for the next. Besides cluster trials, a common answer is the stepped-wedge design: every ward or site eventually gets the tool, but the order and timing of switching over are decided at random. It suits a hospital that plans to roll a tool out everywhere anyway and wants to learn from the rollout.

Accuracy is not the same as benefit. A model can detect disease well and still not improve outcomes, if the finding does not change treatment or if staff do not act on it. Only the upper rungs measure benefit.

Here is an example. A model predicts deterioration on a ward with good accuracy. But most of the patients it flags are ones the nurses were already worried about and watching closely. The model is right, yet it adds little: the extra information arrives when nothing new can be done with it. Only a study in which staff use the model, compared with care without it, can show whether it changes anything.

A further point is which outcome a study measures. Some studies measure a process, such as how quickly antibiotics were given or how many scans were reported within an hour. Others measure what happens to patients, such as deaths, complications or days in hospital. Process measures are quicker and cheaper to collect and often move first, but a faster process does not guarantee better health. When a study reports improvement, check which kind of outcome improved.

Each rung also catches problems the one below cannot. A silent run often shows that data the model expects arrives late or not at all in the live system, for example a blood result that was in the cleaned research dataset but is only filed hours later in practice. A clinical study shows how staff actually respond: whether they read the alert, trust it too much, or start to ignore it. None of this is visible in a retrospective test, however large.

Go deeper: when the right answer is not known for everyone

Every rung of the ladder depends on a reference standard, and in some settings the truth is only checked for some patients. In breast screening, for example, only women who are recalled usually have a biopsy. A cancer missed by both the readers and the AI is not biopsied at the time, so it cannot be counted as missed on the day. Checking the truth only for patients who tested positive is called verification bias, and it tends to make tests look better than they are.

Studies get round this with follow-up: a cancer found within a fixed time after a normal screen counts as a miss. The length of that window changes the answer. Too short, and slow-growing misses have not yet shown up, so sensitivity looks high. Too long, and new cancers that were not visible at screening are counted against the test. Always check how long the follow-up was.

Reporting standards

International groups have agreed what a good AI study must report. You can use their checklists to judge any paper:

  • TRIPOD+AI: for studies that develop or validate prediction models.
  • DECIDE-AI: for early live clinical evaluation.
  • SPIRIT-AI and CONSORT-AI: for protocols and reports of randomised trials of AI.

A reporting standard is a checklist of what a paper should tell the reader, so that someone else can judge the study and, ideally, repeat it. It does not grade the study as good or bad; it makes sure the information you need to judge it is there. A protocol is the plan for a trial, written and published before it starts, so that readers can later check the trial did what it said it would.

You do not need to memorise the checklists. Most of their items come down to questions you can ask of any paper:

  • Where did the patients come from, and how many were left out, and why?
  • How was the reference standard decided, and by whom?
  • How was missing data handled?
  • Was the test data kept separate, by patient, and was any of it from another site?
  • For AI specifically: which version of the model was used, and what did staff see?

The usual mistake is to take a paper's headline number without checking which of these the paper answers. A study that cannot say where its test patients came from cannot support the score it reports.

Two wider cautions apply to the published evidence as a whole. First, studies with impressive results are more likely to be written up and published than studies in which a tool did not help, so the literature on any tool tends to look better than the full picture. Second, many evaluations are run or paid for by the company that makes the tool. That does not make them wrong, but it is a reason to look for independent studies, and to check whether the comparison and the outcome were chosen before the study began, as a published protocol shows.

Those who buy tools for a health service usually need more than accuracy and outcome evidence. They also ask what the tool costs to buy, install and run, how much staff time it saves or uses, and whether the gain is worth the cost compared with other ways of spending the same money. This kind of judgement is called health technology assessment, and it builds on the evidence described in this lesson rather than replacing it.

In England, this is the work of NICE, the National Institute for Health and Care Excellence. Its evidence standards framework for digital health technologies sets out the evidence a tool should have for the level of risk it carries. Its early value assessments look at promising tools while evidence is still being gathered. In January 2025, for example, NICE said that four AI tools that help detect fractures on X-rays in urgent care could be used in the NHS while more evidence is collected.

Worked example · a silent pilot

A deterioration alert runs silently for a month in a hospital with 400 admissions a day. It would have alerted on 8% of admissions, and 20% of those alerts were true deteriorations.

alerts per day = 400 × 8% = 32
true alerts per day = 32 × 20% = about 6
false alerts per day = about 26

Before going live, the team now knows the workload it would create. Twenty-six false alerts a day may be acceptable for a quick check, or overwhelming for a full review. The numbers are for illustration.

Step by step. 8% of 400 is 0.08 × 400 = 32 alerts a day. 20% of those is 0.20 × 32 = 6.4, which rounds to about 6 true alerts. The rest are false: 32 − 6.4 = 25.6, about 26. Over a 30-day month, that is about 960 alerts in all.

What the answer means. "20% of alerts were true" is the positive predictive value from Measuring performance, measured on live patients. It means one alert in five is real, so staff would respond to about five alerts for every true deterioration found. The silent month gives the team a real number to plan with, and a baseline against which later monitoring can be compared. What it cannot show is whether acting on the alerts would help patients; that needs the next rungs.

Turning it into staff time. Suppose each alert takes a nurse about ten minutes to check. Then 32 alerts a day is 32 × 10 = 320 minutes, a little over five hours of nursing time every day, of which about four hours go on the 26 false alerts. Put this way, the decision is easier to discuss with the ward: is finding about six true deteriorations a day worth five hours of checks, and could a higher threshold cut the false alerts without losing too many true ones? The ten minutes is an assumption for illustration; a real pilot would measure it.

Evidence: from validation to trials · LSMA free course Live data from each new patient goes two ways. Staff treat the patient as usual and outcomes are recorded. In parallel, the model scores the patient and its alert is stored but never shown to staff. Later, the stored alerts are compared with the recorded outcomes. SILENT PROSPECTIVE EVALUATION New patient,live record Staff treat as usual,outcomes are recorded staff never see the alert Model scores the patient,alert stored, not shown worked example: 32 alerts a day logged Compare alertswith outcomes
The model runs on real, new patients, but its output goes only into a log. Because nobody acts on it, the comparison later shows how the model performs, and how many alerts it would create, without any risk to patients. Schematic, not to scale.
0.000.250.500.751.0001224364860MonthsSurvival probabilityLow-risk signatureHigh-risk signature
Outcome curves are how clinical studies show whether a tool changes what happens to patients. Illustrative data.

Reading the figure. Each line follows one group of patients over time, for example those cared for with the tool and those without it. The vertical axis shows the proportion still free of the outcome, such as death or admission to intensive care; each step down is a patient having the outcome. If the lines separate, and stay apart, the groups fared differently. In a randomised trial, that difference can be put down to the tool, because chance decided who got it.

The current references are: TRIPOD+AI (Collins and colleagues, BMJ, 2024), which updated the 2015 TRIPOD statement; DECIDE-AI (Vasey and colleagues, 2022, published in Nature Medicine and the BMJ); and SPIRIT-AI and CONSORT-AI (Cruz Rivera, Liu and colleagues, 2020, published in Nature Medicine, the BMJ and Lancet Digital Health). A common error is to cite them as proof that a study is good. They check what was reported, not whether the methods were sound. To judge the risk of bias in a prediction model study, the usual tool is now PROBAST+AI (Moons and colleagues, BMJ, 2025), which updated PROBAST (Wolff and colleagues, Annals of Internal Medicine, 2019) to cover machine learning models.

Key point

Ask which rung a tool's evidence reaches. Most AI tools have internal or external validation only; few have shown improved patient outcomes.

A second example: a vendor reports that its fracture-detection tool "matches specialist radiologists" on 5,000 images from three hospitals. That is rung 2, external validation, and it is useful. It does not tell you whether patients in your emergency department would be treated faster or more safely with the tool, which is the question your hospital is really asking.

Module 2 has given you the tools to judge a model on paper: the right measures, at the right prevalence, from the right kind of study. Module 3, AI in practice, builds on this by following tools out of the study and into everyday care. It starts with AI across the patient journey, which looks at where tools are used today and which rung their evidence has reached, and goes on to deployment, safety and regulation, and the AI assistants many staff already use.

End of Module 2: Judging a modelYou can stop here: this module is complete on its own. Or carry on to Module 3, AI in practice (Advanced).

Further reading · Module 2Trevethan 2017: sensitivity, specificity and predictive values, with their pitfalls. · All reading for this course

Check your understanding

1. In a silent prospective evaluation, what do staff see?

2. Which design best shows a tool improves patient outcomes?

3. Which checklist covers studies that develop or validate prediction models?

Up next

Lesson 1

AI across the patient journey

Ask BioTutor

Create a free website with Framer, the website builder loved by startups, designers and agencies.