Lesson 4 of 7
Module 2 · Statistics
How does birth weight change with the mother's weight, and with smoking, at the same time? Linear regression answers that, with an estimate and an interval for each factor.
Correlation
A correlation coefficient, written r, is a single number between −1 and +1 that measures how closely two measurements follow a straight line. +1 means a perfect rising line, −1 a perfect falling line, and 0 no straight-line relationship at all. The version used here is Pearson's correlation, the most common one. It has no units, so it is the same whether weight is measured in pounds or kilograms.
cor.test(bw$lwt, bw$bwt)
ggplot(bw, aes(x = lwt, y = bwt, colour = smoke)) +
geom_point(alpha = 0.7) +
geom_smooth(method = "lm", se = TRUE) +
scale_colour_manual(values = c("non-smoker" = "#0F1B33", smoker = "#D9481F")) +
labs(x = "Mother's weight (lb)", y = "Birth weight (g)", colour = NULL) +
theme_minimal()What the code does
cor.test(bw$lwt, bw$bwt)calculates the correlation between the mother's weight and the baby's weight, with a confidence interval and a test of whether the true correlation could be zero.- The
ggplot()call draws a scatter plot, one point per baby, coloured by smoking.geom_smooth(method = "lm", se = TRUE)adds a straight fitted line for each colour group, with a grey band showing the 95% confidence interval for where the line lies.
Reading the output. cor = 0.1857 is the estimated correlation. Its 95% confidence interval runs from 0.044 to 0.320, so the true correlation could be anywhere from almost nothing to moderate. t = 2.5848 on df = 187 (the number of pairs minus 2) gives p = 0.0105: if there were no linear relationship, a correlation this far from zero would be unusual.
A correlation of 0.19 is weak but unlikely to be zero. Squared, it says the mother's weight alone accounts for about 3.5% of the variation in birth weight. Correlation measures how closely points follow a straight line, not how steep it is, and not cause.
The step from 0.19 to 3.5% is r squared: 0.1857 × 0.1857 = 0.0345. Two contrasting examples make the point about steepness. If every baby were exactly 1 g heavier for each extra pound of the mother's weight, with no scatter, r would be 1, although the effect is tiny. If birth weight rose steeply with the mother's weight but with a lot of scatter, r could be small. And a strong curved relationship, such as one that rises and then falls, can give r close to 0, which is one more reason to look at the scatter plot first.
The p-value and the strength of a correlation are separate things. The test only asks whether the true correlation could be zero, and the larger the sample, the smaller the correlation it can detect. With these 189 pairs, any r above about 0.14 in size would give p below 0.05; with 1,000 pairs, about 0.06 would. In a large dataset almost every correlation is "significant", so judge it by its size and its interval, not by its stars.
Three pitfalls are worth knowing. A single extreme point, such as one very heavy mother with a heavy baby, can create or hide a correlation on its own. A narrow range hides relationships: had the study included only mothers between 110 and 130 lb, r would probably be much smaller, although the underlying relationship would be the same. And the test assumes that the pairs are independent, one mother and one baby each. Correlation is also symmetric, the same whichever variable you call x, so it cannot say which variable depends on which. Regression, next, makes that choice explicit.
Go deeper: Spearman correlation, the rank-based alternative
Pearson's r uses the actual values, so a few extreme points can raise or lower it a lot, and it measures only straight-line patterns. Spearman's correlation, written ρ (rho), first replaces each variable with its ranks, as the Wilcoxon test did in Comparing two groups, and then calculates Pearson's r on those ranks. It therefore measures whether one variable tends to rise as the other rises, along a straight line or a curve, and an extreme value can shift only one rank.
cor.test(bw$lwt, bw$bwt, method = "spearman") gives ρ = 0.25 (p = 0.0006), a little higher than Pearson's 0.19. R warns that it cannot compute an exact p-value with ties, because several mothers share the same recorded weight; the approximate p-value it gives instead is fine at this sample size. A clear gap between the two coefficients is a hint to look at the scatter plot for outliers or curvature.
Linear regression with several factors
Linear regression describes an outcome as a starting value, the intercept, plus a fixed amount for each unit of each factor, the coefficients or slopes. R chooses the values that make the squared vertical distances between the points and the model's predictions as small as possible, a method called least squares. Each of those vertical distances, observed minus predicted, is a residual. Categorical factors such as smoking enter the model as a shift up or down compared with their reference level.
Regression connects with what you have already done. A model with smoking alone, lm(bwt ~ smoke, data = bw), has a coefficient for smokesmoker equal to the difference in means you found in Comparing two groups, about −284 g. Its p-value matches the classic Student's t-test, which assumes equal spreads, rather than the Welch version R chose by default, so it is close to but not exactly 0.007. Adding more factors to the formula is what lets regression compare smokers and non-smokers who are alike in other respects.
Linear regression needs a numeric outcome, here grams. For a yes-or-no outcome such as low, the logistic regression of Counts and proportions takes its place; the formula and the way you read the coefficients carry over, with odds ratios instead of grams.
lm1 <- lm(bwt ~ lwt + smoke + race, data = bw) summary(lm1) round(confint(lm1), 1)
What this does
lm(bwt ~ lwt + smoke + race, data = bw)fits a linear model for birth weight with three factors at once, and stores it aslm1. The+signs mean each factor gets its own coefficient.summary(lm1)prints the full table of results.round(confint(lm1), 1)gives a 95% confidence interval for every coefficient, to one decimal place.
Reading the output, part by part
- Call repeats the model you fitted.
- Residuals summarises the gaps between real and predicted birth weights. The median, 26 g, is close to zero, and the quartiles (−458 and 476 g) are roughly balanced, which is what a sensible model gives.
- Coefficients has one row per term. Estimate is the coefficient; Std. Error its standard error; t value is the estimate divided by its standard error (for lwt, 3.938 ÷ 1.705 = 2.31); Pr(>|t|) is the p-value for "this coefficient is zero". < 2e-16 means smaller than the smallest value R reports, and the stars are a shorthand explained in the Signif. codes line.
- Residual standard error: 680.3 is the typical size of a residual, in grams. It has 184 degrees of freedom: 189 babies minus 5 estimated coefficients.
- Multiple R-squared: 0.1481 is the share of the variation in birth weight the model explains. Adjusted R-squared, 0.1296, is slightly lower because it allows for the number of terms; adding any factor, even a useless one, raises the ordinary R-squared a little.
- The F-statistic tests whether the model as a whole explains more than an intercept alone would. Its p-value, 5.757e-06, means 0.000005757, very small.
- The last table is the confidence intervals, one row per coefficient.
The intercept, 2,799 g, is the predicted weight for a baby whose mother is in every reference group (non-smoker, recorded as white) and weighs 0 lb. No mother weighs 0 lb, so the intercept is only an anchor for the line, not something to interpret.
You can make the intercept meaningful by centring a factor: subtracting a typical value before fitting. With lm(bwt ~ I(lwt - 130) + smoke + race, data = bw), where I() tells R to do the arithmetic, the intercept becomes the predicted weight for a non-smoking mother recorded as white who weighs 130 lb. That is 3,311 g, the same prediction worked out by hand below. The slope and every other coefficient are unchanged; only the anchor moves.
One more detail of the table: race has two rows, each comparing one group with the reference group. Neither row on its own tests whether race matters overall. drop1(lm1, test = "F") tests each factor as a whole, by asking how much worse the model becomes when that factor is removed.
Worked example · reading the coefficients
smoker −400: 400 g lighter (95% CI 186 to 614 g), with the same weight and recorded race
R² 0.148: the model explains about 15% of the variation in birth weight
Step by step. For the mother's weight, multiply the coefficient by the change: 10 × 3.938 = 39.4 g, so about 39 g; for 30 lb, 30 × 3.938 = 118 g. For smoking, the coefficient is −399.8, so smokers' babies are predicted to be about 400 g lighter, and the interval from confint(), −613.9 to −185.7, becomes "186 to 614 g lighter". A full prediction adds the pieces. For a non-smoking mother recorded as white who weighs 130 lb: 2,799.3 + 130 × 3.938 = 2,799.3 + 511.9 = 3,311 g. For a smoking mother who is otherwise the same: 3,311 − 400 = 2,911 g.
What the answer means. Those predictions are averages for babies like these. With a residual standard error of 680 g, individual babies will often be 680 g or more above or below the prediction. R² of 0.148 can also be read as: 85% of the variation in birth weight comes from things this model does not include.
The smoking effect grows from 284 g (Comparing two groups) to 400 g once the other factors are in the model. Most of the variation is still unexplained, so the model is useful for understanding factors, not for predicting an individual baby's weight.
Why did the smoking effect change from 284 g to 400 g? The unadjusted comparison in Comparing two groups put all smokers against all non-smokers, whatever their weight or recorded race. If smoking was more common in a group whose babies tended to be heavier for other reasons, that group's advantage would partly cancel the harm of smoking in the simple comparison. The model compares smokers and non-smokers within the same weight and race, which removes that offset. This is the same confounding you met with the odds ratio in Counts and proportions.
The model also makes an assumption visible in the figure: the smoking effect is the same, 400 g, at every mother's weight. That is a choice, not a finding. To let the slope differ between smokers and non-smokers, you add an interaction, written lwt * smoke in the formula, which estimates a separate slope for each group. If the interaction is small and uncertain, the simpler parallel-lines model is a fair description; if it is large, a single smoking effect would hide the fact that it depends on the mother's weight.
Check the assumptions
plot(lm1) draws four diagnostic plots. Look for residuals spread evenly around zero with no curve or funnel, and points close to the line in the Q–Q plot. A few large residuals are normal; a clear pattern means the model is missing something.
The four plots are, in order:
- Residuals vs Fitted: residuals against predicted values. A curve suggests the relationship is not a straight line; a funnel, wider at one end, suggests the spread is not constant.
- Normal Q–Q: compares the residuals with what a normal distribution would give. Points near the diagonal line mean the residuals are roughly normal; points curving away at the ends mean heavier tails or skew.
- Scale-Location: another view of whether the spread of the residuals changes with the predicted value.
- Residuals vs Leverage: picks out single observations that have a large pull on the fitted line. Points far outside the dashed Cook's distance lines deserve a closer look.
With 189 babies, mild departures from normality matter little, because the estimates are averages over many values. A clear curve or funnel matters more, because it means the coefficients describe the data poorly in some range.
It helps to list what the model assumes, and which check goes with each. The relationship with each numeric factor is a straight line (plot 1). The spread of the residuals is the same at every predicted value (plots 1 and 3). The residuals are roughly normal (plot 2), which matters most in small samples. And the observations are independent, which no plot can show: it comes from the design, such as one baby per mother. Note that the normality assumption is about the residuals, not the outcome itself. A skewed outcome can give well-behaved residuals once the factors are in the model, and the reverse.
If a check fails, change the model rather than discarding the data. A curve can be handled by adding a curved term or transforming a factor; a funnel often shrinks when the outcome is analysed on a log scale. An influential point should first be checked for a recording error. If it is a genuine value, fit the model with and without it and report whether the conclusion changes.
The case for plotting before trusting a summary is made by Anscombe's "Graphs in statistical analysis" (The American Statistician, 1973). His four small datasets share almost the same means, variances, correlation (about 0.82) and fitted line, yet their scatter plots look completely different; R includes them as anscombe. The Cook's distance lines in the fourth diagnostic plot come from Cook 1977, "Detection of influential observation in linear regression" (Technometrics). Treat these diagnostics as aids to judgement, not as tests with a pass mark: there is no single cut-off that makes a point an error, and an influential point should be checked, not simply deleted.
Key point
A regression coefficient is the effect of one factor with the others held fixed. Always say what else was in the model when you report it.
The smoking effect in this lesson shows why: it is 284 g on its own and 400 g with weight and recorded race in the model. A reader told only "smoking: −400 g" cannot tell which comparison you made. Write instead "400 g lower, adjusted for the mother's weight and recorded race".
Further reading · Module 2Greenland et al. 2016: 25 common misreadings of p-values and confidence intervals. · All reading for this course
Check your understanding
1. The coefficient for lwt is 3.9. A mother 20 lb heavier, all else equal, is expected to have a baby:
2. A correlation of 0.19 means:
3. Why did the smoking effect change when other factors were added?
Up next
Lesson 5
Multiple testing and power