LSMA

LONDON SCHOOL OF MULTIOMICS & AI

Free course

Free course

0 of 73 lessons

Lesson 8 of 9

Module 7 · From genes to function

Microbiomes and metagenomics

Microbiomes and metagenomics

Microbiomes and metagenomics

How to profile a whole community of microbes: 16S amplicon versus shotgun sequencing, taxonomic profiles, richness and Shannon diversity in Python, why shares are not amounts, and how contamination fools low-biomass studies.

⏱ 13 min readModule 7 · From genes to functionFree
Lesson at a glance6 steps · about 13 min
  1. A genome of many genomes
  2. 16S amplicon sequencing
  3. Shotgun metagenomics
  4. From reads to a profile, and to diversity
  5. Proportions can mislead
  6. Contamination

A genome of many genomes

Every lesson so far has studied one organism at a time. A microbiome is the community of microbes living in one place, such as the gut, the skin, soil or sea water, together with their genes. It includes bacteria, archaea, fungi and viruses. A gram of stool holds billions of bacterial cells from hundreds of species, and most of them have never been grown in a laboratory dish.

Metagenomics gets round that problem by sequencing DNA taken straight from the sample, without culturing anything. The word is sometimes used for any sequencing of a community, but strictly it means sequencing all the DNA, and that is how this lesson uses it. The first decision in any microbiome study is which of two approaches to use.

Why does it matter that most gut bacteria have never been grown? Because nearly everything in the earlier lessons of this module assumed a genome you could study on its own. Here, the DNA of hundreds of species arrives mixed in one tube, in unknown proportions, and much of it belongs to species with no reference genome. Every method in this lesson is, in one way or another, a way of sorting that mixture.

16S amplicon sequencing

Every bacterium and archaeon carries a gene for 16S ribosomal RNA, part of the machine that makes proteins. The gene is about 1,500 bases long. Some stretches are almost identical in all bacteria; between them lie nine variable regions, V1 to V9, whose sequence differs between groups of bacteria. Primers that match the conserved stretches copy one or two variable regions by PCR, a reaction that makes millions of copies of a chosen stretch of DNA, often V4 or V3–V4, and the copies, called amplicons, are sequenced.

Identical reads are grouped into amplicon sequence variants (ASVs), exact sequences that each stand for one kind of bacterium. Older studies grouped reads that were at least 97% identical into operational taxonomic units (OTUs). Each ASV is then named by comparing it with a reference database of 16S sequences, such as SILVA or GTDB. A short variable region usually identifies the genus reliably but often not the species.

Grouping reads into ASVs is not as simple as collecting identical sequences, because sequencing errors make some reads from the same bacterium differ by a base or two. Programs such as DADA2 first denoise the reads: they learn the error pattern of the run and work out which rare variants are errors from a common sequence and which are real. Only then are the exact sequences that survive counted as ASVs. The 97% rule for OTUs was a rougher answer to the same problem, merging errors and real variants alike.

Why only the genus? Closely related species can have identical or nearly identical sequences across a 250-base variable region, so there is simply not enough difference to tell them apart. Sequencing the full 1,500-base gene with long reads improves this, but even then some species cannot be separated by 16S at all.

Shotgun metagenomics

In shotgun metagenomics, all the DNA in the sample is broken into fragments and sequenced, as in How sequencing works. Reads come from every gene of every organism present, including viruses and fungi, which 16S misses. Reads can be named by matching their k-mers (Module 6) against reference genomes, as Kraken 2 does, or by looking for marker genes found in only one group of species, as MetaPhlAn does. Because the reads cover whole genomes, shotgun data can also show what the community can do: which genes for digesting fibre or resisting antibiotics are present. Reads can even be assembled into metagenome-assembled genomes, drafts of genomes of microbes nobody has cultured.

Assembling a metagenome is harder than the single-genome assembly of Module 6. Coverage is uneven, because abundant species give many reads and rare ones few; and closely related strains share long stretches of sequence, which tangle the assembly graph in the same way repeats do. After assembly, the contigs must be sorted into genomes, a step called binning. Contigs from the same genome tend to have similar coverage and similar k-mer composition, so binning tools group contigs by both. Each bin is then checked for completeness and contamination, in the same spirit as the completeness checks of Module 6. Genes on the bins can be predicted with the tools of Finding genes and annotating a genome; Prodigal's mode for short or mixed sequences, mentioned there, exists for exactly this case.

16S ampliconShotgun metagenomics
What is sequencedOne gene, copied by PCRAll DNA in the sample
Who is seenBacteria and archaeaAlso viruses, fungi and the host
Usual resolutionGenusSpecies, sometimes strain
FunctionOnly guessed from namesGenes measured directly
Cost per sampleLowHigher; needs more reads
Main biasesPrimer choice; 16S copy numberHost DNA; gaps in reference genomes

How to read this table

Read each row as a trade-off. Two entries need explaining. 16S copy number: different bacteria carry between one and about fifteen copies of the 16S gene, so a species with many copies yields more amplicons and looks more abundant than it is. Host DNA: in a stool sample most DNA is microbial, but in a skin swab or a tissue biopsy most of it can be human, so a shotgun run spends most of its reads on the host. 16S avoids this, because its primers do not copy human DNA. A common plan is 16S for many samples, then shotgun for a chosen subset.

From reads to a profile, and to diversity

Either approach ends in a taxonomic profile: a table with one row per taxon (a named group, such as a genus) and one column per sample, holding read counts. It looks like the gene count table of Module 5, and it raises similar questions. Samples are sequenced to different depths, so raw counts are first turned into relative abundance, each taxon's share of the sample.

Diversity within one sample, called alpha diversity, is summarised in two ways. Richness is the number of taxa present. The Shannon index combines richness with evenness, how equal the shares are: it is higher when there are many taxa in similar amounts, and low when one taxon dominates. Comparing whole communities between samples is beta diversity, often measured with the Bray–Curtis dissimilarity, which runs from 0 (identical) to 1 (nothing shared).

The two kinds answer different questions. Alpha diversity asks how varied each sample is on its own, and is compared between groups, for example "is gut diversity lower after antibiotics?". Beta diversity asks how different the samples are from each other, and is usually shown by placing samples on a plot so that similar communities sit close together. If samples from treated and untreated people form two separate clouds on such a plot, their communities differ, even if their alpha diversity is the same.

The code needs numpy and pandas, both already in Colab. The genus names are real gut bacteria, but the counts are made up for teaching. The table includes a blank: a tube with no sample that went through the same DNA extraction and sequencing.

Python
import numpy as np
import pandas as pd

# Read counts per genus: made up for teaching (three stool samples and a blank)
counts = pd.DataFrame({
    "Stool_A": [4200, 2600, 150, 900, 80, 70, 0],
    "Stool_B": [1500, 1400, 2900, 1200, 600, 400, 0],
    "Stool_C": [9100, 300, 0, 0, 500, 0, 100],
    "Blank":   [12, 0, 0, 0, 3, 0, 85]},
    index=["Bacteroides", "Faecalibacterium", "Prevotella",
           "Bifidobacterium", "Escherichia", "Akkermansia", "Ralstonia"])

print(counts.sum())                                   # reads per sample
rel = counts / counts.sum()                           # each column sums to 1
print((rel * 100).round(1)[["Stool_A", "Stool_B", "Stool_C"]])

def shannon(col):
    p = col[col > 0] / col.sum()                      # skip absent genera
    return -(p * np.log(p)).sum()

summary = pd.DataFrame({"richness": (counts > 0).sum(),
                        "shannon": counts.apply(shannon).round(2)})
print(summary)

What this does

  • counts is a table with genera as rows (index) and samples as columns.
  • counts.sum() adds each column, giving the reads per sample. counts / counts.sum() divides every count by its column total, so each column now adds up to 1. Multiplying by 100 shows percentages.
  • shannon takes one column, keeps the taxa with counts above zero, turns them into proportions p, and returns −Σ p × ln p, using the natural logarithm.
  • (counts > 0).sum() counts the non-zero rows in each column, which is the richness. counts.apply(shannon) runs the function on each column.
Stool_A 8000 Stool_B 8000 Stool_C 10000 Blank 100 dtype: int64 Stool_A Stool_B Stool_C Bacteroides 52.5 18.8 91.0 Faecalibacterium 32.5 17.5 3.0 Prevotella 1.9 36.2 0.0 Bifidobacterium 11.2 15.0 0.0 Escherichia 1.0 7.5 5.0 Akkermansia 0.9 5.0 0.0 Ralstonia 0.0 0.0 1.0 richness shannon Stool_A 6 1.11 Stool_B 6 1.62 Stool_C 4 0.39 Blank 3 0.50

Reading the output

The stool samples have 8,000 to 10,000 reads; the blank has only 100, as expected for a tube with almost no DNA. Stool_A and Stool_B both contain six genera, so their richness is equal, but their Shannon values differ: Stool_B (1.62) is more even, with no genus above 37%, while Stool_A (1.11) is dominated by Bacteroides at 52.5%. Stool_C has the lowest diversity (0.39), with 91% Bacteroides. Now look at Ralstonia: it makes up 85 of the blank's 100 reads and turns up in Stool_C alone. A genus that dominates the blank and appears in only one real sample is a likely contaminant, not a gut resident.

Worked example · Shannon diversity by hand

proportions 0.50, 0.25, 0.25
H = −(0.50 × ln 0.50 + 0.25 × ln 0.25 + 0.25 × ln 0.25)
= −(0.50 × −0.693 + 2 × 0.25 × −1.386) = 0.347 + 0.693 = 1.04
maximum for 3 taxa = ln 3 = 1.10 · evenness = 1.04 ÷ 1.10 = 0.95

A community of three genera, one making up half the reads and two a quarter each.

Step by step. For each taxon, multiply its proportion by the natural logarithm of that proportion. Logarithms of numbers below 1 are negative: ln 0.50 = −0.693 and ln 0.25 = −1.386. So 0.50 × −0.693 = −0.347, and each quarter gives 0.25 × −1.386 = −0.347. The sum is −1.04, and the minus sign in front of the formula makes it 1.04. The highest possible value for three taxa comes when all are equal, ln 3 = 1.10. Dividing gives the evenness, 0.95.

What the answer means. An evenness close to 1 says the three genera are nearly balanced. If one genus made up 98% of the reads, the index would fall towards 0 even though richness was still 3. Shannon values depend on the base of the logarithm, so check which one a paper used before comparing values, and compare only samples processed the same way.

Go deeper: richness depends on how deeply you sequence

Richness counts taxa that were seen, and the more reads you take, the more rare taxa you see. A sample sequenced to 50,000 reads will usually show more genera than the same sample at 5,000. Comparing raw richness between samples of different depth therefore partly compares depth.

Two tools help. A rarefaction curve plots the number of taxa found against the number of reads, by subsampling. If the curve has flattened, more sequencing would add little. If it is still climbing, richness is underestimated. Rarefying means subsampling every sample down to the same number of reads before comparing. It is simple and common, but it throws data away, and some statisticians argue against it for testing differences. Shannon is less sensitive to depth than richness, because rare taxa add little to it, but it is not immune.

Proportions can mislead

A sequencer produces a set number of reads, whatever the amount of bacteria in the sample. So a profile tells you shares, not amounts: the numbers are compositional, meaning they always add up to the same total. If one taxon grows, every other taxon's share must shrink, even if nothing else changed. The code shows this.

Python
before = pd.Series({"Bacteroides": 400, "Escherichia": 100, "Akkermansia": 500})
after = before.copy()
after["Escherichia"] = 4000                           # only one genus grows

print(pd.DataFrame({"before_%": (before / before.sum() * 100).round(1),
                    "after_%": (after / after.sum() * 100).round(1)}))

What this does

  • before holds made-up absolute amounts of three genera, for example cells per gram.
  • after is a copy in which only Escherichia grows, from 100 to 4,000. The other two are unchanged.
  • Both are turned into percentages and shown side by side.
before_% after_% Bacteroides 40.0 8.2 Escherichia 10.0 81.6 Akkermansia 50.0 10.2
Microbiomes and metagenomics · LSMA free course Two stacked bars of relative abundance. Before: Bacteroides 40%, Escherichia 10%, Akkermansia 50%. After only Escherichia grows: Bacteroides 8.2%, Escherichia 81.6%, Akkermansia 10.2%. Side notes say the absolute amounts of Bacteroides and Akkermansia did not change. RELATIVE ABUNDANCE, % BeforeAfter Bacteroides 4010Akkermansia 50 8.2Escherichia 81.610.2 Only Escherichia grew. Bacteroides and Akkermansia stayed exactly the same, yet their shares fell about five-fold. Proportions cannot tell these two stories apart.
Each bar is one sample and adds up to 100%. Grey is Bacteroides, vermilion Escherichia, navy Akkermansia. Values from the code above; amounts made up for teaching.

Reading the output and the figure

Before, Bacteroides is 40% and Akkermansia 50%. After, they are 8.2% and 10.2%, although neither changed at all. Read naively, the profile says two genera crashed. This is why a study can report that a taxon "decreased" when in fact another one bloomed. Three remedies are used. Log-ratio methods, such as the centred log-ratio transform, compare each taxon with the others instead of with the total. Absolute quantification measures total bacterial load, for example by qPCR of the 16S gene or by flow cytometry, and multiplies it back in. Spike-ins add a known amount of foreign DNA to each sample as a ruler. Correlations between proportions are also unreliable, for the same reason; the caution of Biological networks and systems biology applies twice over.

The case that microbiome data must be treated as compositional is set out in Gloor et al. 2017, Frontiers in Microbiology. Vandeputte et al. 2017, Nature, measured total bacterial load by flow cytometry alongside sequencing and found that load varies greatly between people, and that allowing for load changed which taxa appeared linked to other variables. When citing either, note what each fixes. Log-ratio transforms make shares comparable but still cannot recover absolute amounts. They also need a rule for zeros, usually a small added count, and results can depend on that choice. Absolute methods add their own measurement error.

Contamination

DNA extraction kits and laboratory reagents carry traces of bacterial DNA, sometimes called the "kitome". In a stool sample, full of bacterial DNA, these traces are swamped. In a low-biomass sample, one with very little microbial DNA such as blood, tissue or placenta, they can make up most of the reads, and contaminants have been reported as discoveries. Genera such as Ralstonia are repeatedly found in reagents.

  • Negative controls: extraction blanks and PCR blanks, processed alongside the samples and sequenced. Without them, contamination cannot be recognised.
  • Positive controls: a mock community of known bacteria in known amounts, which shows how far the method distorts the truth.
  • Statistical checks: tools such as decontam flag taxa that are more common in blanks, or that rise as the sample's DNA amount falls, which is the signature of a contaminant.
  • Reporting: list which taxa were removed as contaminants and why. A reader cannot judge a low-biomass result without knowing what the blanks contained.
  • Cross-contamination: DNA can splash between neighbouring wells on a plate, so randomise where samples sit.

Key point

A microbiome profile gives shares, not amounts. Choose 16S or shotgun by the question, read diversity as richness plus evenness, treat every rise or fall in a share with care, and never trust a low-biomass result without blanks.

A second example: a study of a tissue thought to be sterile finds bacterial DNA in every sample. If the blanks contain the same genera in similar amounts, and the samples with the least DNA show the most of them, the simplest explanation is reagent contamination. Only a signal clearly above the blanks, ideally confirmed by another method such as microscopy or culture, is worth reporting.

Further reading · Module 7QIIME 2 documentation: tutorials from raw amplicon reads to diversity. · MGnify: browse public microbiome studies and their analysed profiles. · All reading for this course

Check your understanding

1. You need to know which antibiotic resistance genes a gut community carries. Which approach?

2. Two samples both contain six genera, but one has a much higher Shannon index. Why?

3. After treatment, Akkermansia falls from 50% to 10% of reads. What is the safest reading?

Up next

Project

From a gene list to biology

Ask BioTutor

Create a free website with Framer, the website builder loved by startups, designers and agencies.