LSMA

LONDON SCHOOL OF MULTIOMICS & AI

Free course

Free course

0 of 11 lessons

Guided case study

Module 3 · AI in practice · Self-checked

Evaluate an AI tool

Evaluate an AI tool

Evaluate an AI tool

Your hospital has been offered a sepsis early-warning tool. Use everything in the course to decide whether, and how, to adopt it. Write your own answer to each task, then open the model answer and tick it off.

⏱ 11 min readModule 3 · AI in practice · Self-checkedFree
Lesson at a glance8 steps · about 11 min
  1. The situation
  2. Terms in the fact sheet
  3. How to approach this
  4. Define the use
  5. Weigh the evidence
  6. Do the maths for your hospital
  7. People and safety
  8. Fairness, regulation and data

The situation

This case brings the whole course together. A vendor has offered your hospital a sepsis early-warning tool, and the digital board has asked for your view. You have two sources: the vendor's fact sheet, and a short description of your own hospital. Both are fictional, written for teaching; the questions they raise are the ones real evaluations meet.

Read each line of the fact sheet slowly and ask two things of it: what does this line tell me, and which lesson does it connect to? Notice also what is missing. In an evaluation, a line labelled "not reported" can matter as much as any number.

A sepsis tool is a deliberate choice. You met the real-world example in AI across the patient journey, where an independent evaluation found a widely used model much weaker than reported. The same pattern of strong internal figures and missing external evidence is common across many kinds of tool, so the method you practise here carries over to imaging, language and administrative tools alike.

Remember who you are writing for. A digital board is usually a mix of clinicians, managers, information governance and technical staff. They do not need every calculation, but they do need to know what the tool would do on your wards, what is not yet known, and what it would take to find out. Your answers should be short enough to read in a meeting and precise enough to act on.

Vendor fact sheetFictional tool, for teaching
Product
“Vigil”, a sepsis early-warning score that runs on electronic records and alerts the ward team.
Development
One academic hospital, 120,000 admissions from 2019 to 2021. Sepsis in 6% of admissions.
Testing
80/20 random split by admission. AUC 0.88. At the default threshold: sensitivity 85%, specificity 85%.
Not reported
External validation, calibration, results by subgroup, any prospective use.
Regulation
“Registration as a UK medical device in progress.”
Data
Patient records are sent to the vendor's cloud servers outside the UK for scoring.

Terms in the fact sheet

  • Admissions are hospital stays. One patient admitted three times counts as three admissions.
  • An 80/20 random split means 80% of the records were used to build the model and 20% held back to test it, chosen at random. How a medical AI model is built explains why the unit of the split matters.
  • AUC summarises how well the score ranks patients across all thresholds; the default threshold is the cut-off the vendor recommends for raising an alert. Both are from Measuring performance.
  • Cloud servers are computers run by a company and reached over the internet, here located outside the UK.
Your hospital
Admissions
300 a day. Sepsis in about 3% of admissions.

How to approach this

  • Write first, then check. For each task, write your own answer before opening the model answer. The value is in finding out what you would have said on your own.
  • Use the lesson tags. Each task names the lessons it draws on. If you are unsure, reread the relevant part before answering, not after.
  • Keep your numbers. Task 3 produces figures that tasks 4 and 6 use. Write them down with one decimal place.
  • Judge the evidence, not the vendor. The aim is not to prove the tool bad or good, but to say what is known, what is not, and what would settle it.
  • Watch for the gap between "not shown" and "shown not to work". Missing evidence is a reason for caution and for further testing, not proof that the tool fails. A good answer keeps the two apart.
  • A hint sits above each model answer. It points to the method without giving the result. Read it only if you are stuck.
Evaluate an AI tool · LSMA free course The six tasks of the case study in order: define the use, weigh the evidence, do the maths, people and safety, fairness, rules and data, and recommend a decision. Under each is the lesson it draws on. Tasks 1 to 5 feed the recommendation in task 6. SIX TASKS, ONE DECISION 1 Define the use M1 L3 · M3 L3 2 Weigh the evidence M1 L3 · M2 L2 3 Do the maths M2 L1 4 People and safety M3 L1 and L3 5 Fairness, rules, data M1 L2 · M3 L4 6 Recommend a decision all lessons M = module, L = lesson. Each of tasks 1–5 supplies a reason for the decision in task 6.
Work through the tasks in order: each one uses the answers before it, and the recommendation in task 6 should rest on all five. The tags under each box are the lessons to revisit.
1

Define the use

How a medical AI model is built; Deploying AI in a hospital

Write one sentence stating who would receive Vigil's alerts, when, and what they should do.

Hint. Use the three workflow questions from Deploying AI in a hospital: who sees the output, when and where; what they should do with it; and what it adds to their work. A good sentence names a role, the moment an alert fires, the system it appears in, and the action expected, with a time limit.

Check your answer

For example: “Ward nurses receive an alert on the electronic record when an adult inpatient's score passes the threshold, and must review the patient and escalate using the existing sepsis pathway within an agreed time.” Without this, no evaluation can say whether the tool helps.

2

Weigh the evidence

How a medical AI model is built; Evidence: from validation to trials

Which rung of the evidence ladder does Vigil reach? List three weaknesses in how it was tested.

Hint. Put the "Testing" and "Not reported" lines of the fact sheet next to the five rungs in Evidence: from validation to trials. For the weaknesses, ask the questions from How a medical AI model is built: what was the unit of the split, and can one patient have more than one admission? How many sites supplied the data? Then check which of the measures from Measuring performance are absent.

Check your answer

Rung 1, internal validation only. Weaknesses: a split by admission lets the same patient appear in training and testing (leakage); a single hospital, so no external validation; no calibration or subgroup results; no prospective use.

Note also the dates. The data covers 2019 to 2021, years in which hospital care was disrupted by the COVID-19 pandemic, so case mix and recording habits may differ from today's. That is a reason to ask for temporal validation on later patients as well as external validation.

3

Do the maths for your hospital

Measuring performance

Using the vendor's sensitivity and specificity, estimate alerts per day, true alerts per day and PPV at your hospital's 3% prevalence.

Hint. Follow the prevalence example in Measuring performance, but for one day. Split the daily admissions into those with and without sepsis. Apply the sensitivity to the first group to get true alerts, and 1 − specificity to the second to get false alerts. PPV is true alerts divided by all alerts. Then repeat the PPV calculation at the developer's prevalence, to see how the setting changes it.

Check your answer

Sepsis per day: 300 × 3% = 9. True alerts: 9 × 85% ≈ 7.7. Without sepsis: 291; false alerts: 291 × 15% ≈ 43.7. Total about 51 alerts a day, PPV ≈ 7.7 ÷ 51.3 ≈ 15%: roughly 1 in 7 alerts is real. At the developer's 6% prevalence the PPV would be about 27%, so the tool will look worse here than advertised, even before any drop in accuracy.

The negative predictive value tells the other half of the story. Patients without sepsis and without an alert: 291 × 85% ≈ 247.4. Septic patients missed: 9 × 15% ≈ 1.4. NPV ≈ 247.4 ÷ 248.7 ≈ 99.5%. A silent tool is reassuring for any single patient, but across the hospital that still means roughly one septic patient a day with no alert, which is why the absence of an alert must never replace clinical judgement.

One caution about the calculation itself. It assumes the vendor's sensitivity and specificity hold here unchanged, which is the most optimistic case: the figures came from an internal test that may have suffered from leakage, and your patients differ from the development hospital's. Because sepsis is only half as common here, any risk percentages the tool displays are also likely to be too high unless it is recalibrated, which is one reason the missing calibration results matter.

4

People and safety

AI across the patient journey; Deploying AI in a hospital

What risks do 51 alerts a day create for staff and patients? Name two.

Hint. Start from your task 3 numbers: how many of the alerts are false, and how many septic patients get no alert? AI across the patient journey describes what frequent false alarms do to staff; Deploying AI in a hospital describes how people treat a computer's silence as well as its warnings. Think about both the flagged and the unflagged patients.

Check your answer

Alert fatigue: with about 44 false alerts a day, staff may start ignoring alerts, including the real ones. Workload: each alert needs a review, which takes time from other care. Also automation bias: patients without an alert may be assumed safe, though about 1 septic patient a day would be missed.

5

Fairness, regulation and data

Health data: the raw material; Safety, bias and regulation

What would you ask the vendor about fairness, regulation and data before going further?

Hint. Go through the ten questions in Safety, bias and regulation and pick out those the fact sheet leaves unanswered. For safety, recall which clinical safety standard applies to the manufacturer and which to your hospital, from Deploying AI in a hospital. For data, reread the last line of the fact sheet with the data protection paragraph of Safety, bias and regulation in mind.

Check your answer

Fairness: sensitivity and PPV by age, sex, ethnicity and ward. Regulation: is it registered as a medical device for this intended use in the UK, and with what evidence? Safety: their DCB0129 clinical safety case, to support your own DCB0160 assessment, and the DTAC. Data: the legal basis and safeguards for sending records outside the UK under the UK GDPR, and whether scoring could run locally.

6

Make a recommendation

All lessons

Write a three-sentence recommendation to your hospital's digital board.

Hint. One way to structure it: sentence one gives your verdict and the two strongest reasons, using a number from task 3; sentence two says what should happen next, and under which conditions; sentence three says when and how a final decision will be made. Ask yourself which rung of the evidence ladder could be reached safely next, and what must change before it can.

Check your answer

For example: “Vigil is not ready for live use: its evidence is internal only, and at our prevalence it would produce about 51 alerts a day, of which roughly 1 in 7 would be real. We recommend a three-month silent evaluation on our own data, with success criteria agreed in advance (including PPV, subgroup performance and calibration), once registration is complete and data can stay in the UK. We will decide on live use after reviewing those results.”

After you finish

Compare your six answers with the model answers as a set, not one by one. Did your recommendation in task 6 use the numbers from task 3 and the weaknesses from task 2? A recommendation that could have been written without doing the other tasks is too general. If a model answer named something you missed, note which lesson it came from; that is the lesson to revisit.

The same six steps work for any AI tool you are offered, whether it reads scans, drafts letters or predicts readmissions: define the use, weigh the evidence, work out what it would mean at your prevalence, think about the people, check fairness, regulation and data, and only then recommend.

This case also closes the course. Module 1 showed that a model is a set of weights learned from data that records care as well as illness, and that it must be tested on patients it has never seen. Module 2 gave you the measures and the evidence ladder to judge what such tests show. Module 3 followed tools into practice: where they are used, how language models differ, what deployment, monitoring and regulation require, and how to use assistants safely yourself. Each of the six tasks drew on all three. When you are next offered an AI tool, the same questions will serve you, whatever the tool does and however it is described.

Go deeper: how long should the silent evaluation run?

Task 6 proposes a three-month silent evaluation. It is worth asking whether that is long enough. At 9 sepsis cases a day, three months gives about 90 × 9 = 810 cases. A sensitivity measured on 810 cases is fairly precise: if the true value is 85%, the 95% confidence interval, the range that would usually contain it, is roughly 82.5% to 87.5%.

Subgroups are another matter. If one ward or ethnic group contributes only 50 sepsis cases, the same 85% comes with an interval of roughly 75% to 95%, too wide to show whether that group is served as well as others. Agree in advance which subgroups matter and how many cases each needs, and extend the evaluation if needed. Also fix the sepsis definition used to judge alerts before starting, so the result cannot be argued away later.

Take it further

To build and evaluate a model yourself, the next step is to learn a programming language used for data analysis, such as Python or R, and train a simple classifier on a public dataset. Judge it with the measures and checks from this course: a patient-level split, sensitivity and PPV at a realistic prevalence, calibration and subgroup results.

Further reading · Module 3Wong et al. 2021: a widely used sepsis model that performed much worse when tested independently. · All reading for this course

Up next

Finish

Course complete

Ask BioTutor

Create a free website with Framer, the website builder loved by startups, designers and agencies.