LSMA

LONDON SCHOOL OF MULTIOMICS & AI

Free course

Free course

0 of 73 lessons

Lesson 6 of 7

Module 5 · Sequencing data

Beyond RNA-seq: other assays and pipelines

Beyond RNA-seq: other assays and pipelines

Beyond RNA-seq: other assays and pipelines

The same building blocks, quality control, trimming, alignment and counting, run through almost every sequencing assay. What changes is the question, the aligner and what you count.

⏱ 13 min readModule 5 · Sequencing dataFree
Lesson at a glance3 steps · about 13 min
  1. One toolkit, many assays
  2. Workflow managers and community pipelines
  3. Public data

One toolkit, many assays

In the lessons from Quality control to Counting reads per gene you took RNA-seq reads through four steps: a quality check, trimming, alignment and counting. Those steps are not special to RNA-seq. Almost every sequencing assay produces the same kind of raw file, a FASTQ file of reads with a quality score for each letter, so the first steps are shared. What differs is the biological question, and that question decides two things: which aligner (the program that finds where each read came from) you use, and what you count once the reads are in place.

Two kinds of aligner matter most. A splice-aware aligner, such as HISAT2 or STAR, expects that a read from RNA may be split across two exons, because the introns between them were removed when the RNA was spliced. A DNA aligner, such as BWA-MEM or Bowtie2, expects each read to match one continuous stretch of the genome, which is what DNA reads do. Using an RNA aligner on DNA, or a DNA aligner on RNA, gives poor or misleading alignments, so the choice comes first.

The question also decides what is counted at the end, and so what the final table looks like. RNA-seq counts fragments per gene, as in Counting reads per gene. Variant calling looks at each position and asks whether the reads disagree with the reference there. Peak calling looks for regions where reads pile up. The reads, the BAM files and most of the tools in between are the same; the unit of the answer is not.

The good news is that most of what you learned in Aligning reads to a genome carries over unchanged. BWA-MEM and Bowtie2 write the same SAM and BAM formats as HISAT2, so the flags, mapping quality and CIGAR strings mean the same things, and samtools sorts, indexes and summarises them in the same way. The main difference you will see is that DNA alignments have no N in their CIGAR strings, because DNA reads do not skip introns.

Beyond RNA-seq: other assays and pipelines · LSMA free course Raw reads pass through quality checking, trimming and alignment in every assay. After alignment the paths divide: RNA-seq counts reads per gene, genome sequencing calls variants, and ChIP-seq and ATAC-seq call peaks. 1 · SHARED STEPS Raw reads FASTQ files Quality check FastQC Trimming fastp Alignment aligner fits assay 2 · WHAT CHANGES BY ASSAY RNA-seq Aligner: HISAT2 or STAR Result: counts per gene WGS · tumour–normal Aligner: BWA-MEM Result: variant calls ChIP-seq · ATAC-seq Aligner: Bowtie2 Result: peaks
The first three steps are the same for every assay in the table below. The question decides the aligner and what comes out at the end: counts, variants or peaks. Schematic, not to scale.

How to read this table

Each row is one assay. The middle column is the question it is designed to answer; the last column lists the main steps that follow quality control and trimming, in order, joined by arrows. Read along a row and you can see the shared skeleton (align, then clean up, then count or call) with assay-specific tools filled in. You do not need to know every tool yet; the point is to recognise the pattern.

AssayQuestionTypical steps after QC and trimming
Bulk RNA-seqWhich genes change in expression?HISAT2 or STAR (or Salmon without alignment) → counts → DESeq2 or edgeR
Whole-genome or exome sequencingWhich variants does a genome carry?BWA-MEM → mark duplicates → GATK or DeepVariant → annotate variants (VEP)
Tumour–normal sequencingWhich mutations are in the tumour only?BWA-MEM → somatic callers such as Mutect2 → filter and annotate
ChIP-seqWhere does a protein bind DNA?Bowtie2 → remove duplicates → peaks with MACS → differential binding
ATAC-seqWhich chromatin is open?Bowtie2 → remove mitochondrial reads and duplicates → peaks → motifs
Single-cell RNA-seqWhich cell types are present, and what do they express?Cell Ranger or STARsolo → cell-by-gene matrix → Seurat or Scanpy

Terms in the table

  • Salmon without alignment. Salmon does not place each read at an exact genome position. It works out which transcripts each read is compatible with and estimates how many reads came from each transcript. This is much faster than full alignment and is widely used when the goal is expression levels rather than read positions.
  • DESeq2 and edgeR are R packages that test, gene by gene, whether counts differ between groups of samples more than you would expect from variation between replicates.
  • Duplicates are reads that came from copies of the same original DNA fragment, made during the PCR step of library preparation. They are not independent evidence, so in variant calling and peak calling they are marked or removed; otherwise one fragment copied twenty times could look like twenty reads supporting a variant.
  • GATK (the Genome Analysis Toolkit) and DeepVariant are variant callers: they look at all reads covering each position and decide whether the sample differs from the reference there. VEP, Ensembl's Variant Effect Predictor, then says what each variant does, for example whether it changes an amino acid in a protein.
  • Somatic mutations arose in the body during life, here in the tumour; germline variants were inherited and are in every cell. Sequencing normal tissue (often blood) from the same person lets a somatic caller such as Mutect2 set aside the germline variants, which would otherwise swamp the tumour-only changes.
  • Peaks and MACS. A peak is a region where many more reads pile up than in the surroundings. MACS is a widely used peak caller: it decides which pile-ups are higher than expected by chance.
  • Mitochondrial reads in ATAC-seq. ATAC-seq marks open DNA. Mitochondrial DNA is not wrapped around histones, so it is very accessible and can take up a large share of the reads. Those reads say nothing about the nuclear chromatin you are studying, so they are removed.
  • Motifs are short DNA patterns that a particular transcription factor binds. Finding a motif enriched in open regions suggests which factors may be active.
  • Cell Ranger and STARsolo read the cell barcode attached to each single-cell read, so each count is assigned to both a gene and a cell. Seurat (R) and Scanpy (Python) then cluster the cells and find the genes that mark each cluster.

A common mistake is to pick a tool because it appeared in a paper, without checking that it suits the assay. Start from the question, as the table does, and the tool choice follows.

The quality checks from Quality control and Trimming and filtering also apply to every row, but the warning signs differ. A high duplication level is normal for RNA-seq, as Quality control explained, yet in whole-genome sequencing it means the library had too little starting DNA. A high share of mitochondrial reads is a warning in ATAC-seq but tells you little in RNA-seq. Read each QC report against what that assay should look like, not against a fixed list of thresholds.

Go deeper: how duplicates are found, and when to keep them

Duplicate-marking tools such as Picard MarkDuplicates look for read pairs whose two reads start at exactly the same positions, on the same strands. Two independent fragments rarely share both ends, so such pairs are treated as PCR copies of one fragment: the best-quality pair is kept and the others get the duplicate flag, 1024, in column 2 of the BAM file. Some tools also spot optical duplicates, one cluster mistakenly read as two neighbouring ones.

Position alone does not work for RNA-seq, where a highly expressed gene genuinely produces many fragments with the same ends, as Quality control showed. When copies really must be told apart, as in single-cell RNA-seq, each molecule is tagged before PCR with a UMI (unique molecular identifier), a short random barcode. Only reads with the same position and the same UMI then count as duplicates.

Workflow managers and community pipelines

Module 3 introduced workflow managers in Workflows, the cloud and reproducible pipelines. Here you can see why they matter for sequencing data in particular: every sample goes through the same chain of steps, and the number of samples, not the difficulty of any step, is what makes the work hard to manage by hand.

Running each step by hand works for four samples. For hundreds, a workflow manager such as Nextflow or Snakemake runs every step for every sample, restarts from where a run failed, and moves easily from a laptop to a cluster or the cloud. The nf-core community maintains tested Nextflow pipelines for most assays, including nf-core/rnaseq, which does the lessons from Quality control to Counting reads per gene and much more:

Terminal · example
nextflow run nf-core/rnaseq -r 3.14.0 -profile docker \
    --input samplesheet.csv --outdir results \
    --fasta genome.fa --gtf genes.gtf

How a workflow manager helps

A workflow manager works from a description of the steps and the files each step needs and produces. From that it works out the order, runs independent samples side by side, and keeps track of which outputs already exist. If sample 37 of 200 fails because a server ran out of disk space, you fix the problem and resume: the finished work is reused instead of being run again. A cluster is a group of shared computers managed by a scheduler; the workflow manager sends each job to it for you.

The two managers named above describe the same thing in different ways. In Snakemake, written in a language close to Python, each rule says which output file it makes from which input files, and Snakemake works backwards from the files you ask for. In Nextflow, each process is a step, and data flow between processes through channels. Both reach the same result; groups usually choose the one their colleagues already use, which for many sequencing assays means nf-core and therefore Nextflow.

What this does

  • nextflow run nf-core/rnaseq asks Nextflow to fetch the nf-core RNA-seq pipeline and run it.
  • -r 3.14.0 chooses one exact release of the pipeline, so the same tools and settings are used every time you run it.
  • -profile docker tells Nextflow to run each tool inside a Docker container, a packaged copy of the software at a fixed version. You do not need to install each tool yourself, and a colleague gets identical versions.
  • The backslash \ at the end of a line means "the command continues on the next line"; the three lines are one command.
  • --input samplesheet.csv is a table listing each sample and the paths to its FASTQ files. --outdir results is the folder for everything the pipeline writes.
  • --fasta genome.fa and --gtf genes.gtf are the reference genome sequence and the gene annotation, the same two files you used for alignment and counting.

Note that options with one dash (-r, -profile) are read by Nextflow itself, while options with two dashes (--input and the rest) are passed to the pipeline. The run ends with count tables and a combined quality report in the output folder.

Pin the pipeline version (-r) and record it in your methods. Use the nf-core documentation for the current release and the sample sheet format. Learning the steps by hand first, as in this course, is what lets you read and trust a pipeline's output.

For example, if a pipeline reports that one sample aligned at 40% while the others aligned at 95%, you now know where to look: the trimming report for adapter problems, or the FastQC report for a failed run. A pipeline will not decide for you whether that sample should be kept.

Nextflow is described by Di Tommaso et al. (2017, Nature Biotechnology) and Snakemake by Köster and Rahmann (2012, Bioinformatics); the nf-core project is Ewels et al. (2020, Nature Biotechnology). A pipeline is a wrapper around other people's tools, so citing only the pipeline is not enough: nf-core pipelines list the papers for every tool they run, and those should be cited too. In your methods, give the pipeline name and release (for example nf-core/rnaseq 3.14.0), the profile used and any parameters you changed from the defaults. The nf-core website documents each release.

Public data

Most published sequencing data are free to reuse. The Sequence Read Archive (SRA) and the European Nucleotide Archive (ENA) store raw reads; GEO stores processed data such as count tables. For example, a widely used RNA-seq study of airway smooth muscle cells is GEO series GSE52778, with its reads in the SRA under run IDs such as SRR1039508. Ensembl and GENCODE provide matching genomes and annotation.

Reading the identifiers

  • GEO is the Gene Expression Omnibus, run by the US National Center for Biotechnology Information (NCBI). A series (GSE followed by a number) is one study; it lists its samples and links to the raw reads.
  • An SRR number is a run: one set of reads from one sequencing run of one sample. SRR1039508 is one such run. The SRA and ENA share their data, so the same run can usually be downloaded from either.
  • Ensembl and GENCODE release genome sequences and gene annotations in numbered versions. Counts made with one release should be compared with counts made with the same release, because gene models change between releases.

A frequent mistake when reusing data is to download a processed count table and then compare it with your own counts made from a different genome build or annotation release. Check the processing details in the GEO record before you combine anything.

The record also tells you how the reads were made, and you need that before you process them yourself. Check whether the run is single-end or paired-end, the read length, the instrument and, for RNA-seq, whether the library is stranded, which decides the -s setting from Counting reads per gene. Raw reads can be downloaded from the ENA as ready-made FASTQ files, or from the SRA with the fasterq-dump program from NCBI's SRA Toolkit. Either way, run the same QC as for your own data: public data have had no more checking than yours.

Worked example · planning storage and depth

30 million read pairs × 2 reads × 100 bases = 6 billion bases
compressed FASTQ ≈ 0.5 bytes per base → about 3 GB per sample
24 samples → about 72 GB of raw data, before any BAM files

Step by step. A read pair is two reads, one from each end of a DNA fragment, so 30,000,000 pairs is 60,000,000 reads. Each read is 100 bases long: 60,000,000 × 100 = 6,000,000,000 bases. At about 0.5 bytes per base once compressed, that is 6,000,000,000 × 0.5 = 3,000,000,000 bytes, which is 3 GB. For 24 samples: 24 × 3 GB = 72 GB.

Where 0.5 bytes per base comes from. An uncompressed FASTQ file stores one character (one byte) for each base and another for its quality score, plus a header line for every read, so it needs a little over 2 bytes per base. Gzip compression shrinks that to roughly a quarter, giving the rule of thumb used here.

What the answer means. 72 GB is only the starting point. Aligned BAM files, intermediate files and backups come on top, so plan disk space for several times the raw data before you start, and decide early which intermediate files you can delete once the run is finished.

A second calculation. The same arithmetic works in reverse when you plan depth. Whole-genome sequencing is usually described by coverage, the average number of reads covering each base. For a genome of 3 billion bases at 30× coverage you need 3,000,000,000 × 30 = 90 billion bases of reads, which at 0.5 bytes per base is about 45 GB of compressed FASTQ for one person. Coverage returns as a central idea in Module 6, where it is used to estimate the size of a genome no one has sequenced before.

Key point

Whatever the assay, write down the reference version, every tool's version and its settings. That record is what makes a sequencing result reproducible.

A second example: the human reference genome exists in two widely used builds, GRCh37 (also called hg19) and GRCh38. The same variant has different coordinates in each. A variant list that does not say which build it uses cannot be safely compared with any other, however carefully it was made.

Further reading · Module 5Conesa et al. 2016: the whole RNA-seq workflow, from design to interpretation. · All reading for this course

Check your understanding

1. Which aligner is the right choice for whole-genome DNA reads?

2. What does a workflow manager such as Nextflow add?

3. Where would you find a published study's processed count table?

Up next

Project

Reads to a count table

Ask BioTutor

Create a free website with Framer, the website builder loved by startups, designers and agencies.