LSMA

LONDON SCHOOL OF MULTIOMICS & AI

Free course

Free course

0 of 73 lessons

Lesson 2 of 7

Module 2 · Biological data and databases

The main databases and when to use each

The main databases and when to use each

The main databases and when to use each

A handful of public databases answer most questions in biology. Learn the difference between an archive and a curated database, and which one to open for a sequence, a gene, a protein, a structure or a variant.

⏱ 13 min readModule 2 · Biological data and databasesFree
Lesson at a glance4 steps · about 13 min
  1. Archives and curated databases
  2. The international nucleotide archive
  3. Curated sequence and gene resources
  4. Proteins, structures and variants

Archives and curated databases

A database is an organised collection of records that you can search and download. Biology has hundreds of them, but most questions are answered by about ten. The first thing to know about any database is which of two kinds it is.

  • A primary or archival database stores data exactly as researchers submitted it. Each record belongs to its submitter, so the archive keeps it unchanged. That makes archives complete, but also repetitive: the same gene may appear in hundreds of records, some partial, some with errors.
  • A curated (or secondary) database is built from the archives. Curation means experts, helped by computer pipelines, review the evidence, remove duplicates, fix errors and add summaries. The result is smaller, cleaner and usually one record per gene or protein.

Rule of thumb: use the archive when you need everything, including the newest or rarest data; use the curated database when you need one trustworthy answer.

The two kinds do different jobs, so neither replaces the other. An archive is the permanent record of what was measured: if a paper reports a sequence, the archive keeps exactly that sequence, even if a better one appears later, so the paper can always be checked. A curated database is a current best summary, and it changes as evidence grows. A common pitfall is to take the first hit of an archive search as the gene's sequence. It may be a fragment, a variant from one patient, or a record with a sequencing error that was never corrected, because only its submitter may change it.

The international nucleotide archive

Three centres share one archive of DNA and RNA sequences: GenBank, run by NCBI (the US National Center for Biotechnology Information); ENA, the European Nucleotide Archive, run by EMBL-EBI near Cambridge; and DDBJ, the DNA Data Bank of Japan. Together they form the INSDC (International Nucleotide Sequence Database Collaboration). They exchange new records every day, so a sequence submitted to any one of them appears in all three, with the same accession number, the stable identifier of a record. Journals usually require authors to deposit sequences here before a paper is published.

An accession tells you where a record sits but not what it is, so read the record's description before using it. GenBank accessions are short codes of letters and digits, and the version after the dot (as in .1 or .2) changes when the submitter updates the sequence. Many studies also have a BioProject accession, starting PRJNA, PRJEB or PRJDB depending on the centre, that groups all the data from one project; this is often the number a paper quotes in its data availability statement.

Raw sequencing reads, the FASTQ files of Module 1, are far too bulky for ordinary records. They go to companion read archives: the SRA (Sequence Read Archive) at NCBI and the read section of ENA, again mirrored between centres.

Read archives organise data in layers: a study, the samples in it, the experiments done on each sample, and the runs, which are the actual files of reads. Run accessions start SRR, ERR or DRR, depending on which centre received them, and a run accession is what download tools ask for. Raw reads are the most complete form of the data, but also the most work: a single study can be terabytes, so check whether a processed form, such as a count table, already exists before downloading everything.

Go deeper: why there are three centres, not one

The three archives grew up separately. The EMBL Data Library began in Heidelberg in 1980, GenBank at Los Alamos in the USA in 1982, and DDBJ at the National Institute of Genetics in Mishima, Japan, in 1986. GenBank moved to NCBI in 1992. Rather than merge, they agreed to share: each centre accepts submissions from its own region, and all three exchange records daily.

The arrangement has practical benefits. No single country controls the world's sequence record. If one centre has an outage or loses funding, two complete copies remain. And researchers can submit in their own time zone, with local help. For you, the result is simple: an accession number found in any of the three works in all of them, and you can cite it without saying which site you used.

Curated sequence and gene resources

RefSeq is NCBI's curated collection: one reference record for each chromosome, transcript and protein of well-studied organisms. Its accessions show the molecule type in the prefix: NM_ for an mRNA, NP_ for a protein, NC_ for a complete chromosome. Prefixes XM_ and XP_ mark records predicted by computer rather than reviewed. The number after the dot is the version: it goes up each time the sequence changes. For TP53, the reference mRNA is NM_000546 and its protein NP_000537.

The prefixes save time once you know them. One more is worth knowing here: NG_ marks a genomic region, used as a stable reference for a single gene. The underscore is the quick sign of a RefSeq record; plain GenBank accessions never contain one. A frequent mistake is to describe a variant against an XM_ record. Predicted transcripts can change or vanish between releases, so for reports and papers use an NM_ record with its version.

NCBI Gene is a gene-centred page that gathers everything NCBI knows about one gene: its official symbol, location, RefSeq transcripts, function summary, linked variants and papers. Each gene has a plain number as its identifier; TP53 is Gene ID 7157.

NCBI Gene is often the best first stop, because it answers "is this the gene I think it is?" in one page: the official symbol, other names it has had, the species and the chromosome. From there, links lead out to RefSeq, ClinVar, PubMed and the other NCBI resources, so you rarely need to search each one separately.

Ensembl, run by EMBL-EBI, annotates whole genomes, mainly of vertebrates such as human, mouse and zebrafish. Its identifiers start ENSG for genes, ENST for transcripts and ENSP for proteins; TP53 is ENSG00000141510. Ensembl comes with a genome browser, a website that draws a chromosome as a horizontal track and stacks genes, transcripts, variants and other data under it, so you can see what lies in any region. The UCSC Genome Browser does a similar job. For human protein-coding genes, NCBI and Ensembl now agree on one representative transcript per gene, called MANE Select, so that both resources describe variants against the same sequence.

In File formats you will meet you saw GTF and GFF files; Ensembl is one of the main places to download them, together with the matching genome in FASTA. Ensembl numbers its releases, several a year. Because gene models are revised between releases, an analysis should record which release it used, and all its files should come from the same one. For plants, fungi, bacteria and invertebrates, the sister site Ensembl Genomes plays the same role.

Proteins, structures and variants

UniProt is the central protein resource. Its main part, UniProtKB, has two sections. Swiss-Prot is reviewed: curators read the literature and record function, location in the cell, disease links and modifications, each with its evidence. TrEMBL is unreviewed: entries are translated automatically from the nucleotide archive and annotated by computer. Swiss-Prot holds hundreds of thousands of entries; TrEMBL holds hundreds of millions. Human p53 is the reviewed entry P04637.

Two smaller parts of UniProt are useful to recognise. UniParc keeps every protein sequence UniProt has ever seen, without annotation, so an old sequence can always be traced. UniRef groups similar sequences into clusters, which makes large similarity searches faster. For questions about what a protein does, though, start in UniProtKB, and in Swiss-Prot whenever your protein is there.

The PDB (Protein Data Bank) is the single worldwide archive of 3D structures measured by experiment, mainly X-ray crystallography, cryo-electron microscopy (cryo-EM) and NMR. It is run jointly by partner sites in the USA, Europe and Japan, and holds more than 200,000 structures, each with a four-character identifier. Since AI tools learned to predict protein shape from sequence, there is also the AlphaFold Protein Structure Database, run by EMBL-EBI with Google DeepMind, which holds predicted structures for hundreds of millions of proteins. Each prediction comes with a per-residue confidence score called pLDDT, from 0 to 100; low-scoring regions should not be trusted.

Each experimental structure comes with information on how good it is. For X-ray and cryo-EM structures the key number is the resolution, in ångströms: the smaller the number, the finer the detail, so a structure at 1.8 Å shows more than one at 3.5 Å. A PDB entry also often covers only part of a protein, such as one domain, because flexible regions are hard to measure. Before using a structure, check which residues it actually contains.

ClinVar, at NCBI, collects reports from clinical and research labs on whether a human variant causes disease. Each report gives a clinical significance, on a scale from pathogenic through likely pathogenic, uncertain significance and likely benign to benign, and ClinVar shows a review status that tells you how much agreement and expert review lies behind it. Two neighbours are worth knowing: dbSNP gives known variants their rs numbers, and gnomAD shows how common each variant is in large healthy populations.

These resources are strongest when read together. A variant that ClinVar lists as pathogenic for a rare, severe childhood disease, but that gnomAD shows in a few per cent of people, deserves a second look: something that common is unlikely to cause a rare disease on its own. ClinVar also flags conflicting interpretations when labs disagree; such a variant has no settled answer yet, whatever the most recent report says.

DatabaseWhat it holdsUse it when
GenBank · ENA · DDBJEvery submitted DNA and RNA sequence, as submittedYou need all sequences, or a sequence cited in a paper
SRA and ENA readsRaw sequencing readsYou want to reanalyse a study from the start
RefSeqOne curated record per chromosome, transcript and proteinYou need the standard sequence of a gene
NCBI GeneA summary page for each gene, linked to everything elseYou want a quick overview of a gene
EnsemblAnnotated genomes and a genome browserYou want to see a gene in its genomic context
UniProtKB/Swiss-ProtReviewed protein entries with evidenceYou need reliable facts about a protein's function
UniProtKB/TrEMBLUnreviewed, computer-annotated proteinsYour protein is not yet in Swiss-Prot
PDB3D structures measured by experimentYou need a real structure, or a protein bound to a drug
AlphaFold DB3D structures predicted by AI, with confidence scoresNo experimental structure exists
ClinVarLab reports on whether human variants cause diseaseYou need to know if a variant is pathogenic

How to read this table

Read across each row: the name, what is inside, and the kind of question that should send you there. The rows run in the order of the central dogma, from DNA to protein to shape, then to variants. Notice the pairs: GenBank and RefSeq, TrEMBL and Swiss-Prot, PDB and AlphaFold DB. In each pair the first is broader and the second is either more trusted or fills a gap. When two rows could answer your question, the "Use it when" column tells you which to try first.

The main databases: sequences, genomes, proteins, structures · LSMA free course Three rows. Sequences: the shared nucleotide archive of GenBank, ENA and DDBJ, holding every submission, feeds the curated RefSeq and Ensembl records, which are summarised in NCBI Gene. Proteins: unreviewed TrEMBL entries feed reviewed Swiss-Prot entries. Structures: experimental structures in the PDB help train AI predictions held in the AlphaFold database. 1 · BROAD, AS SUBMITTED 2 · CURATED OR PREDICTED Sequences Proteins Structures GenBank · ENA · DDBJ UniProtKB/TrEMBL PDB RefSeq · Ensembl → NCBI Gene UniProtKB/Swiss-Prot AlphaFold DB every submission, many per gene unreviewed, annotated by computer measured by experiment one reviewed set of records per gene reviewed by curators, with evidence predicted by AI, with confidence scores
Each row runs from a broad resource that accepts everything to a smaller one you can trust for a single answer. For structures, the arrow means that experimental structures were used to train the AI that made the predictions. Schematic.

How to read this figure

Read each row from left to right. The left column is broad: it takes everything and checks little. The right column, shaded, is narrower but more reliable or fills a gap. In the top row, curated gene records are built from archive sequences, and NCBI Gene sums them up. In the middle row, a TrEMBL entry can be promoted to Swiss-Prot once a curator has reviewed it. The bottom row is different: an AlphaFold prediction is not a reviewed version of a PDB entry. It is a new kind of record, made by a model that learned from PDB structures, so it must always be read as a prediction, never as a measurement.

Worked example · choosing the right database

A clinician is reading about cystic fibrosis and the gene CFTR. She has three questions.

1. "What is the standard mRNA sequence of CFTR?" → RefSeq (NM_000492)
2. "Is c.1521_1523del (p.Phe508del) reported as disease-causing?" → ClinVar
3. "What does the CFTR protein do, and where in the cell?" → UniProt Swiss-Prot (P13569)

Step by step. For each question, first decide what kind of thing is being asked about: a sequence, a variant or a protein. Then decide whether you need everything or one trusted answer. Question 1 asks for a sequence, and "standard" means one trusted answer, so RefSeq beats the archive, which may hold hundreds of partial CFTR sequences. The NM_ prefix confirms an mRNA. Question 2 asks about a variant's link to disease, which is exactly what ClinVar collects; you would read the clinical significance and check the review status to see how many labs agree. Question 3 asks about protein function, so go to UniProt and pick the reviewed Swiss-Prot entry, whose statements carry evidence, not an unreviewed TrEMBL fragment.

What the answer means. No single database answers everything; each is built for one kind of question. The same gene appears in all of them under different identifiers, which is the subject of Identifiers and cross-references: following one gene across databases. A fourth question, "what does the protein look like?", would go to the PDB first, because a measured structure beats a prediction, and to the AlphaFold database only for regions or proteins with no experimental structure.

Each resource here publishes a reference paper, usually in the database issue of Nucleic Acids Research; for example, the UniProt Consortium updates its description every few years, and ClinVar is described by Landrum and colleagues (2018). The worldwide PDB partnership was announced by Berman, Henrick and Nakamura (2003, Nature Structural Biology). ClinVar's five-step significance scale follows the ACMG/AMP guidelines of Richards and colleagues (2015, Genetics in Medicine).

Database contents change with every release. When you cite a classification or a count, give the release or the date you accessed it: a variant marked uncertain today may be reclassified next year.

Key point

Archives hold everything as submitted; curated databases hold one checked answer. Know which kind you are using, and prefer measured data to predicted data.

A second example: searching for a protein, you find two UniProt entries for the same human gene, one reviewed and one unreviewed, with slightly different sequences. The reviewed Swiss-Prot entry is the one to cite. The unreviewed one may come from a single submitted sequence that carried an error or a rare variant, and no curator has checked it.

Further reading · Module 2INSDC: how GenBank, ENA and DDBJ share one archive. · About RefSeq. · UniProt help. · AlphaFold database FAQ. · ClinVar help. · All reading for this course

Check your understanding

1. A search of the nucleotide archive returns 300 records for one human gene, some partial. Why?

2. You need reliable, evidence-backed facts about what a human protein does. Which source should you prefer?

3. A protein has both an experimental structure in the PDB and an AlphaFold prediction. A low-confidence loop in the prediction differs from the experiment. What is the sensible conclusion?

Up next

Lesson 3

Identifiers and cross-references: following one gene across databases

Ask BioTutor

Create a free website with Framer, the website builder loved by startups, designers and agencies.