LSMA

LONDON SCHOOL OF MULTIOMICS & AI

Free course

Free course

0 of 73 lessons

Lesson 2 of 12

Module 8 · AI for biology

Turning sequences into numbers

Turning sequences into numbers

Turning sequences into numbers

Models only do arithmetic, so every sequence must first become a list of numbers. Build the main feature families from real proteins: one-hot encoding, composition, k-mers, physicochemical properties and evolutionary profiles.

⏱ 16 min readModule 8 · AI for biologyFree
Lesson at a glance5 steps · about 16 min
  1. Why sequences must become numbers
  2. One-hot encoding and composition
  3. Dipeptides and k-mers
  4. Physicochemical properties
  5. Alignment-based features

Why sequences must become numbers

A machine learning model does arithmetic. It multiplies numbers by weights and adds them up, so it cannot read the letters MALWMR directly. Before any model can learn from a protein, the sequence has to be turned into numbers. This step is called feature extraction, or encoding, and the list of numbers it produces for one protein is its feature vector.

There is a second problem. Proteins differ in length: insulin's precursor has 110 residues, while titin has tens of thousands. Most classic models, such as the logistic regression and random forest of A first sequence classifier, evaluated honestly, need every example to have the same number of features. So most encodings either summarise the whole sequence into a fixed-length list, or describe it position by position and leave the length problem to a model that can cope with it, such as the networks of Deep learning on sequences: what the model actually learns and Protein language models and embeddings.

The choice of encoding matters a great deal. A model can only learn from what the features show it. If the features throw away the order of the residues, no model can learn anything that depends on order. This lesson builds the five main families of features, from the simplest to the most informative.

Whatever encoding you choose, it must be applied in exactly the same way to every protein, in training and later when the model is used. Real sequences make this harder than it sounds. UniProt entries can contain X for an unknown residue, U for selenocysteine, or B and Z for residues that could not be told apart. A function that only counts the 20 standard letters will quietly ignore them, so the fractions no longer add up to 1. Decide once how to handle such letters, for example by removing sequences that contain them, and record the choice.

The code needs Biopython. In Colab, run !pip install biopython in the first cell. The sequences are fetched from UniProt, so you get exactly the same ones.

Turning sequences into numbers: features for machine learning · LSMA free course The first six residues of human insulin, M A L W M R, are encoded four ways. One-hot encoding gives a grid with one row per residue and 20 columns, with a single 1 in each row. Composition gives 20 fractions, one per amino acid. Dipeptide counts give 400 numbers, one per pair of neighbours. Physicochemical properties give a few numbers such as hydrophobicity and charge. ONE SEQUENCE, FOUR ENCODINGS M A L W M R … One-hot · length × 20 keeps the order; size grows with length MALWMR Composition · 20 numbers share of each amino acid; order lost Dipeptides · 400 numbers MA, AL, LW, WM, MR …; local order kept Properties · a few numbers hydrophobicity, charge, pI, aromaticity
The same six residues, encoded four ways. Orange squares in the one-hot grid mark the single 1 in each row; the columns follow the alphabetical order of the one-letter codes, ACDEFGHIKLMNPQRSTVWY. Schematic.

How to read this figure

The dark bar at the top is the start of the insulin precursor. The large card on the left is one-hot encoding: one row per residue and one column per amino acid, with a 1 in the column of the residue present and 0 everywhere else. The M row has its 1 in the M column, the A row in the A column, and so on. The three cards on the right each reduce the whole protein to a fixed number of values, however long it is. Read the grey line in each card to see what that encoding keeps and what it loses.

One-hot encoding and composition

One-hot encoding is the most faithful: nothing is lost, and the sequence can be read back from the grid. Its drawback is size. A protein of 110 residues becomes 110 × 20 = 2,200 numbers, and a longer protein becomes more. It is the usual input for deep learning models, which can handle variable length (Deep learning on sequences: what the model actually learns uses it for DNA, with 4 columns instead of 20).

Why not simply number the amino acids, A = 1, C = 2, and so on? Because a model would read those numbers as amounts. It would treat tyrosine (20) as "more" than alanine (1), and cysteine as closer to alanine than to tryptophan, which means nothing chemically. One-hot encoding avoids this: every amino acid is exactly as far from every other, and the model has to learn from data which ones behave alike.

Amino acid composition is the simplest summary: for each of the 20 amino acids, the fraction of the sequence it makes up. It always gives 20 numbers, which add up to 1. It is surprisingly useful. Membrane proteins are rich in hydrophobic residues, and many DNA-binding proteins are rich in positively charged lysine and arginine, and composition picks this up. But it throws away all order: a sequence and its shuffled version have identical composition.

Python
import requests
from Bio import SeqIO
from io import StringIO

def fetch(acc):
    url = f"https://rest.uniprot.org/uniprotkb/{acc}.fasta"
    return str(SeqIO.read(StringIO(requests.get(url).text), "fasta").seq)

ins = fetch("P01308")      # human insulin precursor
hbb = fetch("P68871")      # human haemoglobin subunit beta
print(len(ins), len(hbb))

AA = "ACDEFGHIKLMNPQRSTVWY"
def one_hot(seq):
    return [[1 if aa == a else 0 for a in AA] for aa in seq]
for aa, row in zip(ins[:3], one_hot(ins[:3])):
    print(aa, "".join(map(str, row)))

def composition(seq):
    return {a: seq.count(a) / len(seq) for a in AA}
for name, s in [("insulin", ins), ("HBB", hbb)]:
    top = sorted(composition(s).items(), key=lambda x: -x[1])[:3]
    print(name, [(a, round(f, 3)) for a, f in top])

What this does

  • fetch downloads one UniProt entry in FASTA format, using its accession, and returns the sequence as plain text. requests.get(url).text fetches the file; SeqIO.read reads a FASTA file that holds exactly one record. P01308 is human insulin and P68871 is human haemoglobin subunit beta (HBB).
  • AA fixes the order of the 20 amino acids. Any encoding must use the same order for every protein, or the columns would not line up.
  • one_hot builds one row per residue: for each of the 20 letters, 1 if it matches the residue and 0 if not. The loop prints the rows for the first three residues as strings of digits.
  • composition counts each amino acid with seq.count(a) and divides by the length. The last loop sorts the 20 fractions from largest to smallest and prints the top three for each protein, rounded to three decimals.
110 147 M 00000000001000000000 A 10000000000000000000 L 00000000010000000000 insulin [('L', 0.182), ('G', 0.109), ('A', 0.091)] HBB [('L', 0.122), ('V', 0.122), ('A', 0.102)]

Reading the output

  • 110 147: the insulin precursor has 110 residues and HBB has 147, so their one-hot grids would be 110 × 20 and 147 × 20.
  • The three rows of digits encode M, A and L. Each has a single 1: M in position 11, A in position 1, L in position 10 of the ACDEFGHIKLMNPQRSTVWY order.
  • Leucine is the most common residue in both proteins: 18.2% of insulin's precursor and 12.2% of HBB. Insulin's next are glycine and alanine; HBB's are valine and alanine. Both now have exactly 20 numbers, whatever their length, so they can sit side by side in one table.

Dipeptides and k-mers

To keep some order, count short words instead of single letters. A dipeptide is a pair of neighbouring residues, and there are 20 × 20 = 400 possible pairs. A k-mer is any run of k neighbouring letters; dipeptides are 2-mers. You met k-mers for DNA in Module 6, where the same idea is used to assemble genomes. A sequence of length L contains L − k + 1 overlapping k-mers.

The catch is that the number of possible k-mers grows very fast: 20k. For k = 3 there are 8,000, and most will never appear in a single protein. A table in which most entries are zero is called sparse. Sparse features need far more training examples before a model can learn which words matter.

Raw counts have a second problem: a long protein has more of every word simply because it is long. Dividing each count by the number of words, L − k + 1, turns counts into frequencies that can be compared between proteins of different length, in the same way that composition divides by length. In K-mers and genome size you used k-mer counts on purpose, because there the depth of each word carried information; here it is the relative frequency that matters.

Python
from collections import Counter

def kmer_counts(seq, k):
    return Counter(seq[i:i + k] for i in range(len(seq) - k + 1))

di = kmer_counts(ins, 2)
print(len(di), "different dipeptides out of", 20 ** 2, "possible")
print(di.most_common(3))
print(len(kmer_counts(ins, 3)), "different 3-mers out of", 20 ** 3, "possible")

What this does

  • seq[i:i + k] cuts out the k letters that start at position i. The range stops at len(seq) - k + 1 so the last word does not run off the end.
  • Counter counts how often each word appears. Only words that occur are stored, so the zeros are implied rather than written out.
  • most_common(3) lists the three most frequent dipeptides with their counts. The last line repeats the count for k = 3.
80 different dipeptides out of 400 possible [('AL', 5), ('LL', 3), ('LA', 3)] 102 different 3-mers out of 8000 possible

Reading the output

Insulin's precursor has 109 dipeptides (110 − 1), but they are only 80 different kinds, so 320 of the 400 dipeptide features are zero. The most common pair is AL, five times, followed by LL and LA. Most of these sit in the leucine- and alanine-rich stretch at the start, the signal peptide, a short N-terminal tag that sends a protein for secretion. For 3-mers it is starker: 102 different words out of 8,000 possible, so almost 99% of the vector is zero. That is why 3-mer vectors are rarely used directly for proteins; they are more useful for comparing sequences, as A first sequence classifier, evaluated honestly does.

Go deeper: reduced alphabets: shrinking the k-mer space

One way round sparse k-mers is to use fewer letters. A reduced alphabet groups the 20 amino acids into a handful of classes with similar chemistry, for example hydrophobic, aromatic, positive, negative, small and polar. Each residue is replaced by its class before counting.

With 6 classes, there are 6 × 6 = 36 possible pairs instead of 400, and 6 × 6 × 6 = 216 possible triplets instead of 8,000. A protein of a few hundred residues now fills a far larger share of its 3-mer table, so the features are less sparse and a model needs fewer examples to learn from them. The cost is lost detail: leucine and isoleucine become the same letter, which is usually fine, but so might lysine and arginine, which sometimes is not. Many published groupings exist, and the best one depends on the task.

Physicochemical properties

Letters treat every amino acid as equally different from every other. Chemistry does not: leucine and isoleucine are near twins, while leucine and aspartate are opposites. Physicochemical features replace letters with measured properties. The commonest are:

  • Hydrophobicity, how much a residue avoids water. A widely used scale, the Kyte–Doolittle scale, gives each amino acid a value from −4.5 (arginine, very water-loving) to +4.5 (isoleucine, very water-avoiding). The average over a sequence is called its GRAVY score (grand average of hydropathy).
  • Net charge at a given pH, from the acidic (D, E) and basic (K, R, H) side chains and the two ends of the chain.
  • Isoelectric point (pI), the pH at which the net charge is zero.
  • Aromaticity, the share of the ring-containing residues F, W and Y.

Hundreds of such scales have been published, and collections of them are used to build richer feature sets. Properties can also be computed over a window, such as the first 30 residues, to capture a local signal.

These features sit on very different scales. A pI runs from about 4 to 12, while aromaticity is a fraction, usually below 0.2. Models that add up weighted features, such as logistic regression, are pulled towards whichever feature has the largest numbers, so it is usual to scale each feature first, for example to a mean of 0 and a standard deviation of 1. Tree-based models such as random forests split on one feature at a time and do not need this. Whichever you use, work out the scaling from the training data only and apply the same values to the test data, a point A first sequence classifier, evaluated honestly returns to.

Python
from Bio.SeqUtils.ProtParam import ProteinAnalysis

for name, s in [("insulin", ins), ("HBB", hbb)]:
    p = ProteinAnalysis(s)
    print(name,
          "GRAVY", round(p.gravy(), 3),
          "| charge pH 7", round(p.charge_at_pH(7.0), 1),
          "| pI", round(p.isoelectric_point(), 2),
          "| aromatic", round(p.aromaticity(), 3))

first30 = ProteinAnalysis(ins[:30]).gravy()
print("insulin first 30 residues GRAVY", round(first30, 3))

What this does

  • ProteinAnalysis is Biopython's protein calculator, from Bio.SeqUtils.ProtParam. You give it a sequence and call methods on it.
  • gravy() averages the Kyte–Doolittle values; charge_at_pH(7.0) estimates the net charge at neutral pH; isoelectric_point() finds the pI; aromaticity() gives the share of F, W and Y.
  • The last two lines compute GRAVY for the first 30 residues of insulin only, using the slice ins[:30].
insulin GRAVY 0.193 | charge pH 7 -3.4 | pI 5.22 | aromatic 0.082 HBB GRAVY 0.014 | charge pH 7 -0.7 | pI 6.74 | aromatic 0.088 insulin first 30 residues GRAVY 1.02

Reading the output

  • Both whole proteins have a GRAVY close to zero (0.193 and 0.014), as most soluble proteins do: a mix of water-loving and water-avoiding residues.
  • At pH 7, insulin's precursor carries a net charge of about −3.4 and HBB about −0.7. Their pIs, 5.22 and 6.74, both lie below 7, which is why both are slightly negative at neutral pH.
  • The first 30 residues of insulin give a GRAVY of 1.02, five times the whole-protein value. That is the hydrophobic core of the signal peptide. A whole-protein average hides it; a window feature finds it. A first sequence classifier, evaluated honestly uses exactly this idea.

The hydropathy scale is from Kyte and Doolittle 1982, Journal of Molecular Biology. They proposed averaging it over a window of about 19 residues and suggested that an average above about 1.6 marks a likely membrane-crossing helix. That rule is still a useful first look, but cite it as a historical method. Signal peptides are also hydrophobic and are easily mistaken for membrane helices. Dedicated predictors that model the whole protein, such as TMHMM (Krogh et al. 2001, Journal of Molecular Biology), perform clearly better and are the usual choice today.

Alignment-based features

The richest classic features come from evolution. Search a database for relatives of your protein (with PSI-BLAST, for example, which Module 4 introduces), line them up, and count, at each position, how often each amino acid appears among the relatives. The result is a profile, often stored as a position-specific scoring matrix (PSSM): one row per position and 20 columns, like one-hot encoding, but with scores instead of 0s and 1s. A position where every relative has a cysteine is probably essential; a position where anything goes probably is not.

A PSSM can be summarised into a fixed length, for example by adding up rows that share the same amino acid to get a 20 × 20 table of 400 numbers. Profiles carry far more information than a single sequence and work well for proteins with few close relatives, but they are slow to make: each one needs a database search. You can make one on the NCBI BLAST website by choosing PSI-BLAST under the protein BLAST programme. The language models of Protein language models and embeddings can be thought of as a fast substitute: they learned evolutionary patterns from millions of sequences during training, so they need no search at run time.

A profile is only as good as the relatives it was built from. For a protein from a well-studied family, a search returns hundreds of diverse relatives and the profile is rich. For an orphan protein with few or no relatives, the profile is little more than the sequence itself, and the features add almost nothing over one-hot encoding. When you use profiles, note the database and its release, and how many sequences went into each one, because the same protein searched against a newer, larger database can give a noticeably different profile.

Feature familyNumbers per proteinKeeps order?Typical use
One-hotlength × 20FullyInput to deep learning models
Composition20NoQuick baselines; localisation; small datasets
Dipeptides, k-mers400 for k = 2; 20k in generalLocallyShort peptides; comparing sequences
PhysicochemicalA few, or one per windowOnly through windowsSignals with clear chemistry, such as membrane or secretion
Alignment-based (profiles)length × 20, or 400 summarisedFully, plus evolutionStructure and function prediction; distant relatives

How to read this table

Read across each row. The second column tells you whether the encoding has a fixed length (a plain number) or grows with the protein (length × 20). Fixed-length features suit classic models; growing ones suit deep learning. The third column says how much residue order survives. In practice, people often join several families into one longer vector, called a hybrid feature, for example composition plus a few properties plus a window on the N-terminus. Joining more features gives the model more to work with but also more chances to overfit, so the size of the dataset sets the limit.

Worked example · encoding a five-residue peptide by hand

An example peptide, made up for teaching: MKLLV.

composition: L = 2 ÷ 5 = 0.4; M, K, V = 1 ÷ 5 = 0.2 each; the other 16 = 0
dipeptides: MK, KL, LL, LV → 5 − 1 = 4 words
Kyte–Doolittle: M 1.9, K −3.9, L 3.8, L 3.8, V 4.2
GRAVY = (1.9 − 3.9 + 3.8 + 3.8 + 4.2) ÷ 5 = 9.8 ÷ 5 = 1.96

Step by step. For composition, count each letter and divide by the length, 5. L appears twice, giving 0.4; M, K and V once each, giving 0.2. The four fractions add up to 1.0, and the other 16 amino acids are 0. For dipeptides, slide a window two letters wide along the sequence: MK, KL, LL, LV, which is 5 − 1 = 4 words, each seen once; the other 396 dipeptide features are 0. For GRAVY, look up each residue on the Kyte–Doolittle scale, add the five values to get 9.8, and divide by 5. ProteinAnalysis("MKLLV").gravy() gives the same 1.96.

What the answer means. A GRAVY of 1.96 is strongly hydrophobic, higher even than insulin's signal peptide. A stretch like this, about 20 residues long, could sit in a cell membrane. Notice also what the encodings miss: LVMKL has exactly the same composition and GRAVY, but different dipeptides. Only features that keep order can tell the two apart.

Key point

A model only sees what the features show it. Pick features that can carry the signal you care about: composition for overall make-up, windows for local tags, k-mers for short motifs, profiles for evolution.

A second example: suppose you want to predict whether a protein is phosphorylated at a particular serine. Whole-protein composition is almost useless here, because the signal is the handful of residues around that one serine. A window of, say, seven residues on each side, encoded as one-hot or as properties, gives the model the information it needs.

Further reading · Module 8Biopython Tutorial and Cookbook: the official guide, including sequence handling and protein analysis. · UniProt help: programmatic access: how the REST URLs used here work. · All reading for this course

Check your understanding

1. Two peptides contain exactly the same residues in a different order. Which encoding cannot tell them apart?

2. Insulin's whole-precursor GRAVY is 0.19, but its first 30 residues score 1.02. What does this show?

3. Why are 3-mer counts rarely used directly as features for single proteins?

Up next

Lesson 3

A first sequence classifier, evaluated honestly

Ask BioTutor

Create a free website with Framer, the website builder loved by startups, designers and agencies.