LSMA

LONDON SCHOOL OF MULTIOMICS & AI

Free course

Free course

0 of 35 lessons

Lesson 2 of 9

Module 2 · Figures that tell the truth

Showing distributions

Showing distributions

Showing distributions

A distribution is the shape of the data: where values cluster, how spread they are, whether there are two peaks or outliers. Four common views show different things.

⏱ 10 min readModule 2 · Figures that tell the truthFree
Lesson at a glance3 steps · about 10 min
  1. What this does
  2. Reading the four views
  3. How to read the table below

The code below draws the same 60 tooth lengths from Choosing the right chart in four ways, on one figure, so you can compare what each view shows and what it leaves out. It assumes tg, plt and sns are already loaded from that lesson. If seaborn prints a DeprecationWarning or PendingDeprecationWarning, it comes from the library, not from your code, and you can ignore it.

A distribution is the full set of values a measurement takes, and how often each value occurs. A mean or a median squeezes it into one number and throws the shape away, and the shape is often where the interesting part is: two peaks, a long tail of high values, or one value far from the rest. In Choosing the right chart the question "how are values spread?" pointed to four kinds of chart. Drawing all four for the same data is the quickest way to learn what each one keeps and what it loses.

Python
fig, axes = plt.subplots(2, 2, figsize=(9, 6.5))
sns.histplot(data=tg, x="len", hue="dose", palette="viridis", bins=12, multiple="layer", ax=axes[0, 0])
sns.boxplot(data=tg, x="dose", y="len", color="#DCDFE6", ax=axes[0, 1])
sns.violinplot(data=tg, x="dose", y="len", color="#DCDFE6", inner=None, ax=axes[1, 0])
sns.stripplot(data=tg, x="dose", y="len", color="#0F1B33", size=3.5, ax=axes[1, 0])
sns.stripplot(data=tg, x="dose", y="len", color="#0F1B33", alpha=0.7, jitter=0.15, ax=axes[1, 1])
for ax, t in zip(axes.flat, ["Histogram", "Box plot", "Violin with points", "Every point"]):
    ax.set_title(t, loc="left", fontsize=11)
plt.tight_layout()

What this does

  • plt.subplots(2, 2, …) makes a grid of two rows and two columns. With a grid, each axes is picked by row and column: axes[0, 0] is top left, axes[0, 1] top right, axes[1, 0] bottom left and axes[1, 1] bottom right, counting from 0 as always.
  • Top left: a histogram of length, one colour per dose from the viridis colour scale, with 12 bins. multiple="layer" draws the three histograms over each other rather than stacking them.
  • Top right: a box plot per dose, in a single pale grey.
  • Bottom left: two calls draw on the same axes. First the violins, with inner=None so nothing is drawn inside them; then the points on top, small (size=3.5) and navy.
  • Bottom right: points only. jitter=0.15 spreads them randomly a little to the left and right, so animals with similar lengths do not hide each other; the sideways position means nothing.
  • Titles: axes.flat goes through the four axes in reading order, and zip pairs each one with a title. loc="left" aligns the title to the left edge.

The order of the two calls in the bottom-left panel matters. matplotlib draws each new layer over the ones before it, so the violins go down first and the points land on top, where they can be seen. Swap the two lines and the points disappear behind the grey shapes. The jitter in the bottom-right panel is random, so the sideways position of each dot can change slightly every time you run the code. That is harmless because, as above, the sideways position carries no meaning.

The histogram in the top-left panel is the hardest to read, because three groups are drawn over one another. Two common fixes are element="step", which draws each histogram as an outline instead of filled bars, and a separate small histogram for each dose. With three groups of 20 values, though, the honest answer is that a histogram is not the best tool: it was designed for large samples.

Four views of tooth length by dose: overlapping histograms, box plots, violins with points, and points alone. All show length rising with dose, with similar spread in each group.
The same 60 values four ways, drawn by the Python code above.

Reading the four views

All four agree on the main message: length rises with dose, and the three groups have a similar spread. They differ in what else they let you see. In the histogram the three dose groups overlap and are hard to separate. The box plot gives the clearest summary, but you cannot tell that each box rests on 20 animals. The violin gives a smooth shape, and the points on top show how few values that shape is built from. The points alone show everything, because 60 is a small number.

On real omics data the balance shifts. Draw the expression of 20,000 genes in one sample and the points alone become a solid smear, while the histogram is at its best: it shows that most genes are expressed at low levels and a few at very high levels, a shape called right-skewed, with a long tail to the right. Data like these are usually drawn on a log scale, so that the many small values are not squashed against zero; Honest axes and scales, later in this module, covers when that is fair. The rule is not "always use points" but "choose the view that shows the shape for the number of values you have".

How to read the table below

Each row is one of the four views. "Shows well" is when to choose it; "Hides" is what to add or check when you do. Two terms appear in it. The IQR, interquartile range, is the spread of the middle half of the data: the 75th percentile minus the 25th. A beeswarm plot is a point plot in which points are nudged sideways just enough not to overlap, instead of at random as with jitter.

ViewShows wellHides
HistogramShape, peaks and gaps for large samplesDepends on the bin width; overlapping groups are hard to read
Box plotMedian, middle half (IQR), outliers; compactTwo peaks, and how many points there are
ViolinThe full shape, smoothedSample size; can invent smooth shapes from few points
Points (strip or beeswarm)Everything, for small samplesBecomes cluttered above a few hundred points

The bin width matters because a histogram sorts values into bins before counting. Very wide bins merge real features: two peaks can fall into one bar. Very narrow bins leave only a few values in each, so the bars rise and fall at random and suggest peaks that are not there. The violin's smooth outline has a similar hidden setting, how much it smooths, which is why a violin drawn from five points can look like a confident shape.

In seaborn that setting is bw_adjust: values below 1 smooth less and follow the data more closely, values above 1 smooth more. A second violin setting causes a common pitfall. By default, sns.violinplot extends each outline a little beyond the smallest and largest values, so a violin of lengths can reach below zero, a value no tooth can have. Setting cut=0 stops the outline at the real data range. Whenever a violin's shape surprises you, draw the points on top and check that the shape is really there.

Worked example · reading a box plot

thick line = median · box = 25th to 75th percentile (the IQR)
whiskers = the furthest points within 1.5 × IQR of the box
points beyond the whiskers = possible outliers, shown individually

Say in the legend what the box and whiskers mean: conventions differ between programs.

Step by step. Take an example group whose 25th percentile is 10 and 75th percentile 20. The IQR is 20 − 10 = 10. One and a half times that is 1.5 × 10 = 15. So a whisker can reach down to 10 − 15 = −5 and up to 20 + 15 = 35, and it stops at the furthest real data point inside those limits. A value of 40 lies beyond 35, so it is drawn on its own as a possible outlier.

What the answer means. "Possible outlier" is a drawing rule, not a verdict. A point beyond the whiskers may be a measurement error, or a real and important result, such as one patient who responds very differently. Look at it; do not delete it because the plot marks it.

Showing distributions · LSMA free course Left: a box plot with its parts labelled for an example where the 25th percentile is 10, the median 14 and the 75th percentile 20, so the IQR is 10 and the whiskers can reach from minus 5 to 35; one point at 40 is a possible outlier. Right: twenty values in two separate clusters, around 8 and around 30, drawn as a box plot and as points; the box shows nothing of the two clusters and its median sits in the empty gap. 1 · PARTS OF A BOX PLOT 2 · WHAT A BOX CAN HIDE possible outlier · 40 upper whisker · 32 75th percentile · 20 median · 14 25th percentile · 10 lower whisker · 3 IQR 10 whisker limits: 10 − 15 = −5 and 20 + 15 = 35 no values here box plot the same 20 points
Left: each part of a box plot, using the example numbers above. Right: twenty made-up values in two separate clusters. The box plot looks like an ordinary wide box, and its median falls in the gap where no value lies; only the points show the two clusters. Schematic, not to scale.

The right panel is not a far-fetched case. Groups that hide two clusters are common in biology: two subtypes of a disease, males and females measured together, or two batches of samples processed on different days. Any of these can give a wide, ordinary-looking box. When a box is much wider than you expected, plot the points and colour them by any other variable you have, such as sex or batch, before you interpret the spread.

Go deeper: how many bins?

Rules of thumb can choose a bin count for you. The oldest, Sturges' rule from 1926, uses 1 + log2(n) bins, where n is the number of values. For the 60 tooth lengths that is 1 + 5.9, about 7 bins; for 20,000 genes, about 15. Sturges assumes roughly bell-shaped data and gives too few bins for large samples. The Freedman–Diaconis rule instead sets the bin width from the IQR: 2 × IQR divided by the cube root of n, so a few extreme values do not throw it.

You do not have to work these out by hand. sns.histplot(data=tg, x="len", bins="fd") uses Freedman–Diaconis and bins="sturges" uses Sturges; seaborn passes the name on to NumPy, which does the counting. A good habit is to draw the same histogram with two or three bin settings. A feature that survives every choice is probably real; one that appears at only one setting probably is not.

The box plot was introduced by John Tukey in his book Exploratory Data Analysis (Addison-Wesley, 1977), together with the 1.5 × IQR rule for marking points beyond the whiskers. The violin plot, a box plot with a smoothed outline of the data on each side, was proposed by Hintze & Nelson (1998), "Violin plots: a box plot–density trace synergism", The American Statistician 52(2), 181–184. A short, readable guide is Krzywinski & Altman (2014), "Visualizing samples with box plots", Nature Methods 11, 119–120. Conventions still vary: some programs draw whiskers to the smallest and largest values or to fixed percentiles, and quartiles can be calculated in slightly different ways. matplotlib and seaborn use 1.5 × IQR by default, but say so in the legend.

Key point

Layer the views: a box or violin for the summary, and the points on top for honesty. Hide box-plot outliers (showfliers=False in sns.boxplot) when you also draw all points, so none appear twice.

The right panel of the figure above is the reason. Suppose a gene's expression is measured in 20 tumours and the tumours fall into two subtypes, one with low and one with high expression. A box plot alone shows one wide box and suggests a single, variable group. With the points drawn on top, the two subtypes are plain to see, and the next question, what separates them, becomes obvious.

Here is that layered view for the tooth lengths, as a single panel. It reuses the colours from the figure above.

Python
fig, ax = plt.subplots(figsize=(5, 3.5))
sns.boxplot(data=tg, x="dose", y="len", color="#DCDFE6", showfliers=False, ax=ax)
sns.stripplot(data=tg, x="dose", y="len", color="#0F1B33", alpha=0.7, jitter=0.15, ax=ax)
ax.set_xlabel("Vitamin C dose (mg/day)"); ax.set_ylabel("Tooth length")
sns.despine()

The box gives the summary at a glance, and the points let the reader check it. Say in the legend what the box and whiskers mean and how many values each group holds, here 20 per dose. A distribution describes how individual values vary. It is not the same as how precisely a group's mean is known, which is the subject of the next lesson, Comparisons and uncertainty.

Further reading · Module 2Weissgerber et al. 2015: why bar charts hide the data. · All reading for this course

Check your understanding

1. What can a box plot hide?

2. Why does a histogram's look depend on the bins?

3. You have 5 values per group. Which view is best?

Up next

Lesson 3

Comparisons and uncertainty

Ask BioTutor

Create a free website with Framer, the website builder loved by startups, designers and agencies.