Lesson 3 of 9
Module 3 · Coding with AI assistants
Here is the kind of code an assistant might return for a short, vague request. It looks tidy and it runs. It also contains four bugs. Find them before reading on.
The draft
Imagine you typed "write Python functions for GC content, reverse complement and mean FASTQ quality" and received the file below. It has none of the detail of the prompt template in Asking well: prompts for code: no examples, no edge cases, no mention of case or of the quality encoding. Read it slowly, one function at a time, and write down anything you think could go wrong before you look at the tests.
Reading slowly has a method. For each line, ask two questions: what does this line assume about its input, and what happens if that assumption is false? Then pick a tiny input, two or three bases, and work it through by hand as Python would, one step at a time. This is called tracing the code. It feels slow, but on a function of four lines it takes a minute and finds most of what a test would find.
The trap is reading for shape rather than meaning. Your eye sees a dictionary of base pairs, a join and a return, recognises the pattern of a reverse-complement function, and moves on. The draft below is persuasive precisely because each function has the right shape.
%%writefile ai_draft.py
def gc_content(seq):
gc = seq.count("G") + seq.count("C")
return gc / len(seq) * 100
def reverse_complement(seq):
pairs = {"A": "T", "T": "A", "G": "C", "C": "G"}
return "".join(pairs[b] for b in seq)
def mean_quality(qual):
return sum(ord(c) for c in qual) / len(qual)What this does
%%writefile ai_draft.pyis a Jupyter and Colab cell magic, a special command that starts with %%. Instead of running the cell, it saves the rest of the cell as a file called ai_draft.py, so the functions can be imported later like any module.gc_content(seq)counts the letters "G" and "C" in the sequence with the string methodcount, adds them, divides by the length of the sequence (len(seq)) and multiplies by 100 to give a percentage.reverse_complement(seq)sets up a dictionary, pairs, that maps each base to its partner. It then goes through the sequence base by base, looks up each partner, and joins the results into one string with"".join(...).mean_quality(qual)turns each character of the quality string into a number withord, which returns a character's code (its ASCII value for ordinary characters), adds them up and divides by the number of characters.
Each function is short, named sensibly and formatted neatly. That is exactly what makes a draft like this persuasive: nothing about its appearance suggests a problem.
Tracing the second function on a small input shows how this works. For "AN", the loop takes "A" and finds "T" in pairs; then it takes "N" and looks up pairs["N"]. There is no such key, so Python stops with a KeyError: 'N'. The prompt in Asking well: prompts for code asked for N to stay N; this draft, written from a vaguer request, cannot handle it, and the same happens for any lower-case base.
Check it against answers you know
The quickest test is to run each function on an input whose answer you already know. The four expected values in the comments come from the prompt template in Asking well: prompts for code. The second line uses the same sequence as the first, written in lower case, because DNA from files and databases often comes that way.
from ai_draft import gc_content, reverse_complement, mean_quality
print(gc_content("GGCCAT")) # expect 66.67
print(gc_content("ggccat")) # expect 66.67: same sequence in lower case
print(reverse_complement("AAGC")) # expect GCTT
print(mean_quality("IIII")) # expect 40Reading the output
- 66.66666666666666 is correct: 4 of the 6 bases are G or C. The long tail of sixes is normal: computers store decimals in binary and cannot hold two-thirds exactly, so the last digit is slightly off. You would round it for display, and a test should allow for it.
- 0.0 is wrong. The same sequence in lower case reports no G or C at all.
- TTCG is wrong. It is the complement of AAGC, but not reversed.
- 73.0 is wrong. It is the raw character code of "I", not the quality score it stands for.
Notice how the inputs were chosen. Each is short enough to check by hand, and each tests one thing: the normal case, then a single change (lower case) that real files often contain. When a check fails, you know which assumption broke. Two more inputs are worth trying: reverse_complement("ATGN"), which stops with the KeyError described above, and gc_content(""), which stops with a ZeroDivisionError. An error is a result too: it tells you what the code does not yet handle.
The expected values must come from you, not from the assistant. If you ask the same assistant what mean_quality("IIII") should return, it may give the same wrong answer its code gives, because both come from the same patterns. A check is only useful if its answer was worked out independently of the code being checked.
Worked example · the four bugs
count("G") misses “g” → 0.0 instead of 66.672. reverse complement is not reversed: it returns only the complement, TTCG
3. quality forgets the Phred+33 offset → 73 instead of 40
4. an empty sequence divides by zero and crashes, instead of a clear error
Step by step. (1) In "ggccat", count("G") and count("C") both find 0, because Python treats "g" and "G" as different characters; 0 ÷ 6 × 100 = 0.0. (2) Complementing AAGC base by base gives TTCG; reading TTCG backwards gives GCTT, but the draft never reverses. (3) "I" has code 73; without subtracting 33, the mean of four 73s is 73.0 instead of 40. (4) For an empty string, len(seq) is 0, and dividing by zero raises ZeroDivisionError, a message that says nothing about the real problem, an empty sequence.
What the answer means. The fourth bug is the only one that announces itself, and even then with a misleading message. The other three would flow into tables and figures. A lower-case FASTA file, for example, would report a GC content of 0% for every sequence, and the wrong strand would be passed to a primer design or alignment step without any warning.
Only one of the four checks passed. None of these bugs gives an error on ordinary input: each silently returns a wrong number, which is the worst kind of bug.
Two studies show why reading and testing matter. Pearce et al. (2022), “Asleep at the keyboard? Assessing the security of GitHub Copilot’s code contributions”, IEEE Symposium on Security and Privacy, found that about 40% of the programs Copilot produced in their security-relevant scenarios had a known kind of weakness. Perry et al. (2023), “Do users write more insecure code with AI assistants?”, ACM CCS, found that participants with an assistant wrote less secure code on most tasks and were more likely to believe it was secure.
Both studies tested particular tasks with tools from 2021 and 2022, so do not quote their percentages as true of today’s tools. The finding that lasts is about people: fluent code makes us check less.
A review checklist
Use these questions on every piece of code an assistant gives you, and on your own code too.
- Does it do what I asked, for the exact input I will give it?
- Edge cases: empty, lower case, missing values, one item, very large input.
- Do the functions and arguments it uses really exist, in my version?
- Is anything hard-coded that should not be: file paths, column positions, thresholds?
- Can I explain every line? If not, ask, and check the explanation.
Each question targets a common failure. The first catches code that solves a slightly different problem from yours. The second catches the bugs you have seen above. The third catches invented or outdated functions: look the function up in the documentation for the version you have installed. The fourth catches code that works only on the example it was written for, such as a path like /Users/someone/data.csv that exists only on another computer, or a column picked by position that moves when the file changes. The last is the most demanding and the most useful: a line you cannot explain is a line you cannot check.
Applied to the draft, the checklist finds every bug above in a few minutes. The first question fails for reverse_complement, which does not reverse. The second fails for case, for N and for empty input. The third passes: count, ord and join are all real and long-standing. The fourth finds nothing hard-coded here, but in a longer draft it would catch an unexplained number, such as an offset of 64 written straight into the code. The fifth makes you say what ord returns, and once you say it out loud the missing 33 is hard to miss.
The same questions scale up. When a coding agent changes several files, review the diff, the list of lines that were added, removed or changed, rather than rereading whole files. Read the changed lines with the checklist in hand, and be most careful with deletions: removed lines, such as a check that used to reject bad input, are easy to overlook because there is nothing new to read.
Go deeper: checking a function exists in your version
The third checklist question is quick to answer in code. import pandas as pd; print(pd.__version__) tells you which version you have, and help(pd.DataFrame.map) shows the documentation for that function in that version. If the name does not exist, Python raises an AttributeError at once, which is a useful result in itself.
Outdated suggestions are common because pandas has changed a lot. DataFrame.append was removed in pandas 2.0, so code that uses it now fails; the modern way is pd.concat. DataFrame.applymap was deprecated in pandas 2.1 and renamed DataFrame.map, so older code gives a warning. A deprecated function still works for now but is due to be removed. Treat a deprecation warning in AI-written code as a sign it was learned from older examples.
Key point
Test AI code on a few inputs where you know the answer, including awkward ones, before you trust it with real data.
A second example. A function that merges two sample sheets by sample ID might run perfectly on your test files, where every ID appears in both. Then try one sample missing from the second sheet, and one ID typed with a trailing space. If the function silently drops those samples, you have found the problem on a three-row test instead of in a finished analysis.
Further reading · Module 3Wilson et al. 2014: testing and review habits that apply to AI-written code too. · All reading for this course
Check your understanding
1. Why did gc_content("ggccat") return 0?
2. Which bug would comparing with Biopython catch most directly?
3. What makes these bugs dangerous?
Up next
Lesson 4
Testing AI-written code