Lesson 5 of 6
Module 3 · AI in practice
Many staff already use chat assistants to draft emails, summarise papers or reword a leaflet. A few habits keep patients, colleagues and you safe, and make the output worth using.
Start with the right tool
In Large language models in medicine you saw how these assistants produce likely text rather than checked text, and in Safety, bias and regulation how data protection applies to health information. This lesson turns both into habits you can use the next time you open an assistant at work.
Public chat tools and the versions your organisation has approved can have very different terms: where text is stored, who can see it, and whether it is used to train future models. Before you use any assistant for work, find out which tools are approved and what each may be used for. If there is no approved tool, treat every assistant as public.
The terms of a tool are the conditions you accept when you use it: how long your text is kept, in which country it is stored, whether the provider's staff may read it, and whether it may be used to improve the provider's models. An approved tool is one your organisation has checked and agreed terms for, usually with limits written into a contract. The same product can come in both forms, for example a free personal account and an organisational version, so the name of the tool does not settle the question; the account you are using does.
Approval usually comes with a stated purpose: a tool may be approved for drafting general documents but not for patient information. The usual mistake is to assume that approval for one task covers every task.
Tools also change. A provider may add features, such as connecting to your email or files, or change its terms, and an approval given last year may not cover what the tool does today. Browser extensions and features built into other software, such as a writing aid inside a word processor, can send text to an assistant without it being obvious. If you are not sure whether a feature sends your text elsewhere, treat it as if it does until your organisation says otherwise.
What you may paste
How to read the table below. The columns run from the most sensitive information on the left to the least sensitive on the right. Each row is a different kind of material, so read across a row to see where, for example, a whole letter sits compared with a published leaflet. When something does not obviously fit, treat it as belonging in the column further to the left.
The table sorts by what the material contains, not by where it came from. An internal document may sit in the right-hand column if it says nothing confidential, and a single line of a published report may move to the left if you have added a patient's details to it. A common pitfall is the screenshot: a picture of a screen can carry names, numbers and record details at the edges that were never meant to be part of the question.
| Never, in any public tool | Only in an approved tool, and only if needed | Usually fine |
|---|---|---|
| Names, NHS or hospital numbers, dates of birth, addresses, photos | Clinical text with identifiers removed, when the task needs it | Published guidelines, leaflets and papers |
| Whole letters, notes or records | Internal documents your policy allows | Your own draft text with nothing confidential in it |
| Passwords, access codes, unpublished results, contracts | Anonymised data summaries, such as counts and averages | General questions that mention no one |
Under UK data protection law, health information is special category data: sharing it needs a lawful basis, and a public chat tool rarely provides one. Your organisation's information governance team can tell you what applies to you.
Three terms here. Special category data is the law's name for kinds of personal information that need extra protection, and health is one of them. A lawful basis is one of the legal grounds on which an organisation may use personal data at all, such as providing care. Information governance is the team, and the set of policies, that decide how your organisation handles information. Note also that removing a name does not by itself make data anonymous: if the person could still be identified from what is left, it is still personal data in the eyes of the law.
Lawful use also depends on purpose. Information shared with you to care for a patient is not automatically available for other uses, such as testing a new tool or preparing a talk. Even inside an approved tool, ask whether the task you have in mind is one the information may be used for. When in doubt, use invented or published examples instead; for most drafting tasks they work just as well.
The legal basis for this table is the UK GDPR. Article 9 lists health data as special category data, needing both a lawful basis and an extra condition before it can be used. Recital 26 explains that pseudonymised data, where identifiers are replaced by codes that someone could still link back, is still personal data. Only truly anonymous data falls outside the law. The test is whether someone could identify a person using means reasonably likely to be used, so it depends on context and cannot be settled by a checklist. In the NHS, the Caldicott Principles, updated to eight in 2020, add a duty of confidentiality. This summary is not legal advice; your information governance team decides how it applies.
Removing identifiers is more than removing the name
A rare condition, an exact age, a small town and a date can identify someone together, even with no name. Keep only what the task needs.
Each detail on its own may match thousands of people. Together they narrow the field fast. "A woman in her eighties" fits a great many people; "an 84-year-old woman from a small town, admitted on a named day with a hip fracture" may fit only one, and a neighbour, relative or colleague reading it could recognise her. Working out who someone is from details like these is called re-identification. The safest question is not "which details can I remove?" but "which details does this task actually need?".
Free text is where identifiers hide, as Health data: the raw material showed. A letter may mention a relative by name, an employer, a hospital number in a sentence, or a phrase such as "the patient we discussed at last week's meeting" that colleagues would recognise at once. Reading a passage once from start to finish, looking only for such details, catches far more than deleting the fields at the top. Better still, write a fresh request in your own words that describes the task without the original text.
Worked example · making a request safe
✓ “An older woman admitted with a hip fracture…”
removed: name, exact age, town, NHS number, date → 5 identifiers; the task, a plain-language discharge leaflet, still works
The name and number in this example are invented. Ask first: does the assistant need to know anything about this person at all? For a leaflet, it does not.
Step by step. Go through the first request detail by detail and ask of each one: does the leaflet need it? The name: no. The age 84: no, though "older" helps set the tone. The town: no. The NHS number: never. The admission date: no. The hip fracture: yes, because the leaflet is about recovering from one. That leaves one detail kept and five removed.
What the answer means. The safe request loses nothing the task needs. In fact, a leaflet about recovery from a hip fracture can be written with no patient in it at all, and then reused for every patient. If a task really does need details about one person, such as drafting a letter about their care, that is a job for an approved tool only, and even then only the details the letter needs.
Go deeper: how few details it takes
Re-identification has been measured. In work published in 2000, Latanya Sweeney estimated from US census data that about 87% of Americans could be uniquely identified by just three details: their five-digit postcode, their sex and their full date of birth. None of those is a name.
Later work went further. Rocher, Hendrickx and de Montjoye (Nature Communications, 2019) estimated that 99.98% of Americans could be correctly re-identified in any dataset using 15 demographic details, even when the dataset was only a small sample of the population. Health records are rich in exactly such details: dates of admission, rare diagnoses, ages and places. That is why "keep only what the task needs" protects people better than trying to list every identifier to delete.
Ask in a way that keeps you in control
- Give it the source. Paste the approved leaflet or guideline and ask it to work from that, rather than from what it “knows”.
- Say who it is for and what you want back: “rewrite for patients at a reading age of about 9; keep every instruction; list anything you changed”.
- Ask it to flag uncertainty and to say when something is not in the source.
Each habit closes a gap described in Large language models in medicine. Giving it the source means the assistant can copy from checked text instead of producing what is merely likely. Saying who it is for gives it a clear target: a reading age is the school age at which a typical child could read a text comfortably, and many patient materials aim for a low reading age so that most adults can follow them. Asking for a list of changes gives you a ready-made checklist for your own review. Asking it to say when something is not in the source makes additions easier to spot, though you should still check, because it may not always do so.
Two further habits help. First, ask for one task at a time: a request to summarise, simplify and translate in one step makes errors harder to trace to their cause. Second, keep a short request you know works for a task you do often, such as rewriting a leaflet, and reuse it, so you are not changing two things at once when the result looks different. Neither habit replaces checking; both make the checking easier.
Check before anything leaves your hands
- Every fact, number, dose and date against a trusted source.
- Every reference: look it up. Assistants can invent papers that sound real.
- That nothing important was dropped, softened or added.
- Tone and reading level, for anything a patient will read.
A trusted source is one your organisation relies on for that kind of fact: the original guideline, a drug formulary, the patient's record. Asking the assistant whether its own answer is right is not a check, because it uses the same process that produced the answer. "Softened" deserves a moment's thought: "stop taking this medicine and contact your GP if you get a rash" rewritten as "let your GP know if you notice a rash" reads more gently, and has lost the instruction to stop the medicine.
How carefully you check should match what is at stake, but never fall to zero. A summary of a published paper for your own reading needs a look at the key claims against the paper. A leaflet a patient will follow, or a letter that will sit in a record, needs every instruction compared with its source. The pitfall described in Large language models in medicine applies here too: as an assistant keeps getting things right, it is tempting to check less, and that is exactly when an error is most likely to slip through.
Worked example · checking a rewritten leaflet
An assistant rewrites a medicines leaflet in plain language. A nurse checks each of its 14 instructions against the original.
needing correction = 1 of 14 = 7%, and that one could harm a patient
An illustrative example. The output read well throughout, which is exactly why the error was easy to miss.
Step by step. Check the total first: 12 + 1 + 1 = 14 instructions. One needs correcting: 1 ÷ 14 = 0.071, about 7%. The reworded one counts as correct, because its meaning matches the original; checking means comparing meaning, not wording.
What the answer means. A 7% error rate sounds small, but the error is the kind that matters: some medicines are absorbed differently, or irritate the stomach, depending on whether they are taken with food. A single changed word reversed the instruction. The only way the nurse found it was by checking all 14 instructions, including the 13 that were fine. That is what "every line checked" costs, and why it is still faster than writing the leaflet from scratch.
Record it, and know when not to use it
Follow your organisation's policy on saying when AI helped draft something, and keep a short note of the tool, the task and what you checked. Whatever the tool wrote, the person who signs or sends it is responsible for it.
Do not use an assistant when you cannot check the answer, for a decision about a patient that you have not verified yourself, or when your policy does not allow it.
A note can be one line, for example: "Leaflet rewritten with the approved assistant; all 14 instructions checked against the original by the ward nurse; one corrected." If a question arises later, that line shows what was done and by whom. The second paragraph above is a useful test before you start: if you could not tell a right answer from a wrong one, the assistant cannot save you time, because you would have to find the answer elsewhere anyway.
If you do find an error that could have caused harm, or you realise identifiable information went into a tool that was not approved for it, report it through your organisation's usual incident route, even if no harm resulted. These reports are how organisations learn which tasks and tools need tighter rules, in the same way that the monitoring described in Deploying AI in a hospital catches problems with clinical tools.
Key point
Approved tool, no identifiers, your source in, every line checked out. If you would not email it to a stranger, do not paste it into a public assistant.
A second example: a manager wants help summarising a staff survey. The published national guidance it refers to is fine to paste. The free-text comments, which may name colleagues or describe identifiable incidents, are not, unless the tool is approved for that use. Counts and percentages from the survey, with no comments attached, sit in between: suitable for an approved tool when the summary needs them.
Further reading · Module 3Thirunavukarasu et al. 2023: what large language models can and cannot do in medicine. · All reading for this course
Check your understanding
1. Which request is safe for a public chat tool?
2. An assistant gives three references for a claim. What do you do?
3. Who is responsible for a letter an assistant drafted and you sent?
Up next
Case study
Evaluate an AI tool