Lesson 3 of 9
Module 2 · Figures that tell the truth
An error bar can show three different things, and readers cannot tell which unless you say so. This lesson computes all three, then replaces the bar chart with something more honest.
Before you start. This lesson carries on in the notebook from Choosing the right chart and uses its tg table and okabe colours. If you open a new notebook, or Colab has restarted, run this setup cell first.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
tg = pd.read_csv("toothgrowth.csv")
tg["supp"] = tg["supp"].map({"OJ": "Orange juice", "VC": "Ascorbic acid"})
okabe = {"Orange juice": "#E69F00", "Ascorbic acid": "#0072B2"}Three kinds of error bar
An error bar is a line drawn above and below a summary value, usually a mean. The three common kinds answer different questions:
- The standard deviation (SD) describes the data: roughly how far a typical individual lies from the mean.
- The standard error of the mean (SE) describes the estimate: how much the mean itself would vary if you repeated the experiment with new animals. It is the SD divided by the square root of the number of individuals, n.
- A 95% confidence interval (CI) is a range built so that, if the experiment were repeated many times, 95% of such ranges would contain the true mean. For small samples it is the SE multiplied by a value from the t distribution.
The worked example computes all three for one group of ten guinea pigs from Choosing the right chart.
Which one to show depends on the question. If the reader needs to know how much individuals differ, for example how variable patients' responses are, show the SD, or better the points themselves. If the reader needs to compare group means, show a confidence interval, because it is the one bar that says directly how well each mean is known. The SE on its own is rarely the right choice: it is a step on the way to the CI, and a reader has to multiply it in their head to get anything useful.
A common pitfall is to draw error bars for very small groups without thinking about what they mean. Many lab experiments have three replicates per condition. With n = 3 there are 2 degrees of freedom, and the t value is 4.303, so the 95% CI is more than four SEs wide on each side. That width is not a flaw in the figure; it is an honest picture of how little three measurements can pin down a mean. With so few values, draw all three points as well.
Worked example · three error bars from the same data
± SD 4.46 → how much animals vary
± SE 1.41 = 4.46 ÷ √10 → how precisely the mean is known
± 95% CI 3.19 = 1.41 × 2.262 (t value for 9 degrees of freedom) → a range likely to contain the true mean
SE bars are the shortest, which is why they are popular, and the least useful on their own. Show the points and a 95% confidence interval, and say so in the legend.
Step by step. The square root of 10 is about 3.162, so the SE is 4.46 ÷ 3.162 = 1.41. The degrees of freedom here are n − 1 = 10 − 1 = 9, and the t value that leaves 2.5% in each tail for 9 degrees of freedom is 2.262. So the half-width of the CI is 1.41 × 2.262 = 3.19, and the interval runs from 13.23 − 3.19 = 10.04 to 13.23 + 3.19 = 16.42.
What the answer means. The same ten animals give bars of 4.46, 1.41 or 3.19 on either side of the mean, depending only on which is chosen. A reader who is not told which cannot interpret the figure. The t value is used instead of the familiar 1.96 because with only ten animals the SD is itself uncertain, so the interval must be wider; as n grows, the t value moves closer to 1.96. Note also that the SE shrinks as n grows, while the SD does not: with 40 animals instead of 10, √40 is twice √10, so the SE would be about half as large, but animals would vary just as much as before.
d05 = tg[tg["dose"] == 0.5]
summ = d05.groupby("supp")["len"].agg(n="count", mean="mean", sd="std")
summ["se"] = summ["sd"] / np.sqrt(summ["n"])
summ["ci95"] = summ["se"] * 2.262 # t value for 9 degrees of freedom
print(summ.round(2))
fig, axes = plt.subplots(1, 2, figsize=(8, 3.5), sharey=True)
axes[0].bar(summ.index, summ["mean"], yerr=summ["se"], color=["#0072B2", "#E69F00"], capsize=6)
axes[0].set_title("Bar of means ± SE", loc="left")
sns.stripplot(data=d05, x="supp", y="len", hue="supp", palette=okabe, alpha=0.7, ax=axes[1], legend=False)
axes[1].errorbar(x=[0, 1], y=summ["mean"], yerr=summ["ci95"], fmt="o", color="black", capsize=6)
axes[1].set_title("Every point, mean and 95% CI", loc="left")
for ax in axes: ax.set_xlabel(""); ax.set_ylabel("Tooth length (dose 0.5)")
sns.despine()What this does
d05 = tg[tg["dose"] == 0.5]keeps only the 20 animals on the lowest dose, ten per supplement..agg(n="count", mean="mean", sd="std")computes three summaries per supplement and names the resulting columnsn,meanandsd. pandas'stdis the sample standard deviation, which divides by n − 1.summ["se"] = …adds a column: the SD divided by the square root of n, using NumPy'snp.sqrt. The next line multiplies by 2.262 to get the CI half-width. That number is correct only for n = 10; with a different group size you would need a different t value, and the comment records the assumption.plt.subplots(1, 2, …, sharey=True)gives two plots side by side with the same y axis, so the two panels can be compared fairly.- On the left,
axes[0].bardraws one bar per supplement at the mean, with SE whiskers (yerr) and small caps at their ends (capsize=6). The two colours follow the row order ofsumm, ascorbic acid then orange juice, so they match theokabecolours from Choosing the right chart. - On the right,
sns.stripplotdraws every animal, andaxes[1].errorbaradds a black dot at each mean (fmt="o") with the 95% CI as its bars. The x positions 0 and 1 are where seaborn places the first and second group. - The loop clears the x labels and sets the same y label on both panels;
sns.despine()removes the top and right borders.
The fixed x positions 0 and 1 hide a pitfall. seaborn places text categories in the order they first appear in the data, while groupby sorts them alphabetically. Here the two orders happen to agree, because ascorbic acid rows come first in the file. If the file were sorted the other way, each mean would be drawn over the wrong group's points, with no error message. Passing order=["Ascorbic acid", "Orange juice"] to sns.stripplot makes the order explicit, so it no longer depends on the file.
seaborn can also draw a confidence interval for you, with errorbar=("ci", 95) in sns.pointplot. It computes the interval by the bootstrap, explained in the box below, rather than from the t value, so its numbers will be close to, but not exactly, the ones in the table. Computing the interval yourself, as here, means you know exactly what each bar is and can write it in the legend.
Reading the output
One row per supplement, sorted alphabetically. Both groups have 10 animals. Orange juice has the higher mean, 13.23 against 7.98, and also the larger spread: SD 4.46 against 2.75. Every other column follows from those two: the ascorbic acid SE is 2.75 ÷ 3.162 = 0.87, and its CI half-width is 0.87 × 2.262, about 1.96. The ascorbic acid interval therefore runs from about 6.0 to 9.9, and the orange juice interval from 10.04 to 16.42; the two do not overlap.
Reading the figure
The left panel invites the reader to compare two bar heights and two short whiskers. The right panel shows the ten animals in each group, so the difference in spread and the single high orange juice value are visible, and the longer CI bars say honestly how well each mean is known. Both panels use the same numbers; only the right one lets a reader check them.
When you read someone else's figure, ask three questions before the comparison itself. What kind of bar is it: SD, SE or CI? How many individuals are in each group? Are the individual values shown? If the legend answers none of them, the figure cannot support a strong claim, however clean it looks.
Cumming, Fidler & Vaux (2007), "Error bars in experimental biology", Journal of Cell Biology 177(1), 7–11, is the standard short reference for this section. It separates descriptive bars (SD) from inferential bars (SE and CI), and warns that rules of thumb about overlapping bars depend on the type of bar and on n. In particular, the 95% CIs of two independent groups can overlap a little while the groups still differ at p < 0.05, so overlap is not a test. Cumming & Finch (2005), "Inference by eye", American Psychologist 60(2), 170–180, gives the details. You can check the t value of 2.262 with SciPy: stats.t.ppf(0.975, 9), after from scipy import stats.
Two measures: a scatter plot with a fitted line
When both measures are continuous, each individual becomes one point, placed by its two values. Here each point is one baby: its mother's weight along x and its birth weight up y, coloured by whether the mother smoked.
The data are 189 births recorded at one US hospital, a standard teaching dataset like the tooth growth data. Before fitting anything, look at the cloud of points and ask four things: which way it slopes, how tightly the points follow that slope, whether it bends, and whether a few points sit far from the rest. A fitted line answers only the first question, and only well if the cloud does not bend.
bw = pd.read_csv("birthweight.csv")
g = sns.lmplot(data=bw, x="lwt", y="bwt", hue="smoke",
palette={"non-smoker": "#0072B2", "smoker": "#E69F00"},
height=3.5, aspect=1.3, scatter_kws={"alpha": 0.6, "s": 20})
g.set_axis_labels("Mother's weight (lb)", "Birth weight (g)")What this does
sns.lmplotdraws a scatter plot and fits a straight line through the points by linear regression, the line that best predicts y from x. Withhue="smoke", it fits a separate line for each group.palette={…}gives the two groups their colours.height=3.5is the plot's height in inches, andaspect=1.3makes the width 1.3 times the height.scatter_kwspasses settings to the points only:alphamakes them see-through andssets their size.lmplotbuilds its own figure, likeclustermapin Module 1, and returns an objectg;g.set_axis_labelssets both axis labels, with units: pounds for the mother's weight, grams for the baby's.
You will build this plot again in Multi-panel figures. The shaded band is the 95% confidence interval for the fitted line, not for individual babies.
The difference matters. The band shows where the true average line probably lies; individual babies scatter much more widely around it, so many points fall outside the band. The band is also narrowest near the middle of the data and widens towards the ends, where there are fewer mothers to pin the line down.
Two pitfalls are common with fitted lines. First, a straight line can be fitted through any cloud, even a curved one, so the line alone never proves that the relation is straight; the points underneath are the check. Second, when the two groups have separate lines, readers tend to compare the slopes. Each slope is estimated from only part of the data, so its band is wider than it would be for one line through everyone, and two lines that look different may well be consistent with the same slope. Where the two bands overlap heavily, describe the slopes as similar rather than different. As in the first section, overlap is a rough guide here, not a test.
Go deeper: where the shaded band comes from
seaborn does not draw the band from a formula; it uses the bootstrap. It makes a new sample of the same size from your rows, drawing at random with replacement, so some births are picked twice and others not at all. It fits the line again, and repeats this 1,000 times by default. The band covers the middle 95% of those 1,000 lines at each point along x. Because the resampling is random, the band can shift very slightly each time you run the cell; sns.regplot(data=bw, x="lwt", y="bwt", seed=1) makes it repeatable, just as the seed did for the simulated data in Module 1.
The band is a picture, not a number. For the line itself, use NumPy: slope, intercept = np.polyfit(bw["lwt"], bw["bwt"], 1), where the 1 asks for a straight line. The slope is the change in birth weight, in grams, for each extra pound of the mother's weight. It describes an association in these 189 births, not a cause.
Key point
Bar charts are for counts and proportions, where the bar's length from zero means something. For continuous measurements, show the points.
A fair use of a bar: the share of samples that passed quality control in each of three sequencing runs, say 90%, 75% and 95%. Each bar's length from zero is the quantity itself, and a bar twice as long means twice the share. This is also why a bar chart's axis must start at zero: if it started at 70%, the 75% bar would be only a fifth the length of the 95% bar, when the true ratio is about four-fifths.
Further reading · Module 2Weissgerber et al. 2015: why bar charts hide the data. · All reading for this course
Check your understanding
1. Which error bar describes how much individuals vary?
2. SD is 6 and n is 9. What is the SE?
3. When is a bar chart the right choice?
Up next
Lesson 4
Colour that works for everyone