Lesson 1 of 7
Module 5 · Sequencing data
Every decision in an analysis depends on how the reads were made. This lesson follows RNA from a tube to a FASTQ file, sets up the tools, and reads the first numbers from the practice data.
Module 4, Comparing sequences, worked with a few sequences at a time: aligning two, scoring the match and searching a database with one query. A sequencing run gives millions of short sequences at once, each only a fragment of a molecule, and every one has to be checked and placed before it means anything. This module takes you from those raw reads to a table of reads per gene, using the command line from Module 3. The alignment ideas from Module 4 return in Aligning reads to a genome, scaled up to millions of reads.
How Illumina sequencing makes reads
- Library preparation. RNA is converted to DNA and cut into fragments of a few hundred bases. Short adapter sequences are joined to both ends.
- Clusters. Each fragment is copied many times in one spot on a flow cell.
- Sequencing by synthesis. One base is added per cycle, each with a coloured label. A camera records the colour, so 100 cycles give a 100-base read.
- Paired-end reads. The fragment is read from both ends: read 1 (_R1) and read 2 (_R2). The two reads of a pair come from the same fragment.
What happens at each step
- Library preparation. A library is the whole collection of prepared DNA fragments from one sample. RNA cannot be sequenced directly on this machine, so an enzyme called reverse transcriptase copies it into cDNA (complementary DNA). Most of the RNA in a cell is ribosomal RNA, so kits usually first either pull out messenger RNA by its poly-A tail or remove the ribosomal RNA. The adapters are short pieces of known DNA: they let each fragment attach to the flow cell, give the sequencing reaction a place to start, and often carry an index, a short barcode that lets many samples share one run and be sorted apart afterwards.
- Clusters. The flow cell is a glass slide whose surface is coated with short DNA strands that match the adapters. One fragment binds, and is copied in place until about a thousand identical copies sit together in a tiny spot, a cluster. One molecule gives too faint a signal for a camera; a thousand copies glowing together can be seen.
- Sequencing by synthesis. In each cycle, the machine adds bases that each carry a coloured, removable label and a block that stops the strand growing further. Every copy in a cluster takes exactly one base, the camera photographs the colour, and then the label and block are removed for the next cycle. Over many cycles a few copies in each cluster fall out of step with the rest, so the colour becomes less clear and the quality scores fall towards the end of the read.
- Paired-end reads. Reading from both ends gives two pieces of information from one fragment, and the aligner knows the two reads must land close together and facing each other. If a fragment is shorter than 200 bases, the two 100-base reads overlap in the middle; if it is shorter than 100 bases, each read runs past the end of the fragment into the adapter, the effect you counted in Module 3.
Each of these steps leaves a mark in the data, and the next lessons look for those marks. Adapters show up as adapter sequence at the ends of short-fragment reads, which Trimming and filtering removes. The loss of step in sequencing by synthesis shows up as quality falling along the read, which Quality control measures. Library preparation also copies fragments by PCR (the polymerase chain reaction) before they reach the flow cell, so some reads are exact copies of each other; Quality control calls these duplicates. Knowing where each pattern comes from tells you whether it is a fault or simply how the method works.
The reads arrive as FASTQ files, the format you summarised with a script in Module 3: four lines per read, giving its name, its bases, a + separator and one quality character per base. Read 1 and read 2 are in two separate files, in the same order, so the tenth read in the _R1 file is the partner of the tenth read in the _R2 file. Every tool in this module relies on that order, which is why the two files are always processed together.
Go deeper: two colours, and the trouble with G
Many recent Illumina instruments, such as the NextSeq and NovaSeq, use two colours instead of four. T gives a green signal, C a red one, A both at once, and G neither: a G is recognised by the absence of light. This saves imaging time, but it has a side effect. When a cluster stops giving a signal, for example because its copies have fallen badly out of step, the camera sees darkness, and darkness reads as G.
The result in real data is a run of Gs, often dozens long, at the end of some reads: a poly-G tail. It is not in your sample. fastp, the trimmer in Trimming and filtering, recognises data from these instruments and trims poly-G tails by default. If long runs of G turn up among FastQC's overrepresented sequences, this is the usual explanation.
Set up the tools
Create one conda environment for the course (see Installing software with conda), make a project folder, and put the course files in data/.
The environment for this module holds everything the next four lessons use, so you install once and then only activate it.
conda create -n hts -c conda-forge -c bioconda seqkit fastqc fastp hisat2 samtools subread
conda activate hts
mkdir -p hts-project/{data,qc,trimmed,index,aligned,counts}
cd hts-project # then copy the course files into data/What this does
conda create -n htsbuilds an environment called hts (high-throughput sequencing) with six packages: seqkit for summaries, FastQC for quality reports (Quality control), fastp for trimming (Trimming and filtering), HISAT2 for alignment and samtools for handling aligned reads (Aligning reads to a genome), and Subread, the package that containsfeatureCounts(Counting reads per gene).conda activate htsswitches to it.mkdir -p hts-project/{data,qc,…}makes the project folder and six folders inside it in one go. The shell expands the words in curly braces into six separate paths, one per name.cd hts-projectmoves into the project. Every later command in this module is run from here.
The folders follow the order of the analysis: raw reads in data, quality reports in qc, cleaned reads in trimmed, the genome index in index, alignments in aligned and gene counts in counts. Each step reads from one folder and writes to the next, so you can always see how far a sample has got.
Installing everything at once also means conda chooses versions of the six tools that work together. Once the analysis is finished, conda env export > environment.yml writes the exact version of every package to a file. Keep that file with your results: the version of a tool is part of the method, and defaults sometimes change between releases. A common pitfall is to install tools one at a time into the base environment over several months, until no one can say which version produced which result.
A first look at the data
seqkit stats data/*_R1.fastq.gz
Reading the output
- Each row is the read 1 file of one sample. Because read 1 and read 2 come in pairs, num_seqs here is also the number of read pairs, the number of fragments sequenced.
- sum_len is reads × 100: for example 11,000 × 100 = 1,100,000 bases for ctrl_2.
- The three length columns are all 100: every read is the full length of the run, as expected before trimming.
- Run the same command on data/*_R2.fastq.gz and the number of reads for each sample should match read 1 exactly. If the two files of a pair disagree, one of them was cut short, often by an incomplete download, and the reads are no longer in step. Check this before anything else, because an aligner given mismatched files either stops with an error or pairs the wrong reads.
- seqkit reads the compressed .fastq.gz files directly. There is no need to unzip them, and keeping them compressed saves a great deal of disk space on real data.
Four samples of 100-base paired reads, with different depths: 8,500 to 11,000 pairs. The practice files are tiny so everything runs in seconds; a real RNA-seq sample has 20 to 50 million read pairs, and the same commands work on it.
The number of reads per sample is called its depth (or library size). Depth always varies between samples, because it is hard to load exactly equal amounts of every library onto the machine. Here ctrl_2 has 11,000 ÷ 8,500 ≈ 1.3 times as many reads as treat_2, so every gene in it will tend to have about 1.3 times as many reads for that reason alone. Correcting for this is part of normalisation, which you meet in Counting reads per gene.
Worked example · quality scores
Q30 → 1 in 1,000 · Q20 → 1 in 100 · Q10 → 1 in 10
Step by step. The character I is stored as ASCII code 73. Subtract the offset of 33: 73 − 33 = 40, so the score is Q40. The error chance is 10 to the power of −Q/10: −40 ÷ 10 = −4, and 10−4 = 1 ÷ 10,000. The same arithmetic gives the other three: Q30 is 10−3, Q20 is 10−2 and Q10 is 10−1.
What the answer means. Every 10 points of Q is a tenfold change in accuracy. A Q40 base is wrong once in 10,000 calls; a Q10 base once in every 10, which is too unreliable to trust. In Trimming and filtering you will see the practice data report 98.0% of read 1 bases at Q30 or above.
A run is usually judged by its share of bases at Q30 or above. Quality typically falls towards the end of each read, which is why Trimming and filtering trims read ends.
The Q score scale comes from Phred, a base-calling program for the older Sanger sequencers, described by Ewing and Green (1998, Genome Research). The FASTQ format, and the different offsets used over the years, are set out by Cock et al. (2010, Nucleic Acids Research). Until about 2011 some Illumina software wrote scores with an offset of 64 instead of 33, so very old files can be misread; modern files use 33. The sequencing chemistry is described by Bentley et al. (2008, Nature). When you quote quality figures, remember that a Q score is the instrument's own estimate of error, not a measured error rate, and say which instrument and software produced the reads.
Design comes first
No pipeline rescues a poor design. For differential expression, biological replicates matter far more than depth: three or more per group, and more for subtle effects. Process the groups together, so that batch and treatment are not confused, and record every sample's details in a sample sheet.
A biological replicate is an independent sample of the same condition: a different mouse, patient or culture flask. A technical replicate is the same sample measured twice. Only biological replicates tell you how much genes vary naturally between individuals, and that natural variation is what a treatment effect has to stand out from. Sequencing one sample very deeply measures that one sample very precisely, but says nothing about the next one.
A batch effect is a difference caused by how samples were handled, such as the day of extraction, the kit lot or the sequencing run. If every control was prepared on Monday and every treated sample on Tuesday, then treatment and day are confounded: any difference could come from either, and no analysis can separate them. Mixing the groups across days and runs keeps them apart. The sample sheet is a table with one row per sample, listing its name, its group and every handling detail, so these effects can be checked later.
Depth still matters, but in a different way. A gene with only a handful of reads is measured noisily, because a count of 3 against 6 could easily be chance. Extra depth helps with lowly expressed genes; extra replicates help with every gene, because they tell you how much each one varies. Once most genes of interest have enough reads, money spent on another replicate usually gains more than the same money spent on deeper sequencing.
Give the samples file names that match the sample sheet exactly, as the practice data do: ctrl_1, treat_2 and so on. Every later step builds file names from these, as the loops in this module will show, and a mismatch between the sheet and the files is one of the most common ways that two samples get swapped without anyone noticing.
Key point
The practice data have two replicates per group so they stay small. For a real study, plan at least three, and decide the analysis before you sequence.
To see why three matters, imagine two replicates of a gene that disagree, one high and one low. With only two, you cannot tell which is typical. With three, if two agree, the odd one stands out, and the statistics have a better estimate of how much the gene varies. Deciding the analysis in advance, including which groups you will compare, also stops you from trying comparisons until one happens to look significant.
Further reading · Module 5Conesa et al. 2016: the whole RNA-seq workflow, from design to interpretation. · All reading for this course
Check your understanding
1. In paired-end sequencing, what are the R1 and R2 reads of a pair?
2. A base has quality Q20. What is its chance of being wrong?
3. With a fixed budget, what usually improves a differential expression study most?
Up next
Lesson 2
Quality control