Lesson 3 of 5
Module 1 · R basics
Biological data comes as tables. This lesson covers R's main structures for holding it, and how to read a real sample sheet from a file.
Matrices and lists
A matrix is a grid of values of one type. A count table, with genes in rows and samples in columns, is a matrix. A list holds items of different types together, like a labelled box.
A matrix has two dimensions: rows and columns. Because every value in it has the same type, usually numbers, R can do arithmetic on a whole matrix at once, for example dividing every count by its sample's total. That is why count tables are kept as matrices. A list makes no such promise: one item can be a word, the next a number, the next a whole table. Lists are how R packages bundle related things together, and many analysis results you will meet later are lists underneath.
In Vectors, functions and loops every vector held values of a single type, and a matrix keeps that rule in two dimensions. Square brackets work as they did there, but with two indexes separated by a comma, rows first and columns second: m["MYC", "S2"] gives 61, m["MYC", ] gives MYC's counts in both samples, and dim(m) gives the size as 4 rows and 2 columns.
m <- matrix(c(120, 340, 56, 980, 98, 410, 61, 1015), nrow = 4,
dimnames = list(c("TP53", "BRCA1", "MYC", "GAPDH"), c("S1", "S2")))
m
colSums(m) # total reads per sample
info <- list(dataset = "airway", samples = 8, treatments = c("untrt", "trt"))
info$treatmentsWhat this does, line by line
matrix(c(…), nrow = 4, …)takes eight numbers and arranges them in 4 rows; R works out that eight values in four rows means two columns. R fills a matrix column by column: the first four numbers (120, 340, 56, 980) become the first column, and the next four (98, 410, 61, 1015) the second.dimnames = list(…, …)gives the names: the first item is the row names (four genes), the second the column names (two samples). It is itself a list, because it holds two vectors of different lengths.mprints the matrix.colSums(m)adds up each column, giving the total reads per sample.rowSums()does the same for rows.list(dataset = "airway", samples = 8, treatments = c("untrt", "trt"))makes a list of three named items: a piece of text, a number and a vector of two labels.info$treatmentsuses$to take one item out of the list by its name.
Reading the output. The matrix prints with gene names down the left and sample names across the top. Under it, the column totals are 1496 for S1 (120 + 340 + 56 + 980) and 1584 for S2 (98 + 410 + 61 + 1015). Totals like these are the library sizes you will meet in RNA-seq: S2 has slightly more reads overall, so some of its higher counts reflect depth, not biology. The last line is the two treatment labels, printed with quotes because they are text.
Correcting for library size is the obvious next step, and it holds a trap. m / colSums(m) looks as though it divides each column by its own total, but it does not. R divides element by element and recycles the short vector of two totals down the columns, so the first column is divided alternately by 1496 and 1584. The result has the right shape and plausible numbers, and is wrong. sweep(m, 2, colSums(m), "/") does what you meant: the 2 says "work along columns", and each column is divided by its own total. Afterwards each column sums to 1. Errors that give no error message are the dangerous ones, so check a result like this with colSums() before you use it.
Go deeper: when one column stops being a matrix
Taking a single column from a matrix with m[, 1] does not give a one-column matrix. R simplifies the result to a plain vector and drops the second dimension. That is often convenient, but code that expects a matrix then fails: ncol() returns NULL and colSums() stops with an error.
Adding drop = FALSE keeps the shape: m[, 1, drop = FALSE] is a 4 × 1 matrix that still has its row and column names. This matters most inside functions and loops, where the number of columns you select can change. A filter that keeps two samples returns a matrix, but one that happens to keep a single sample silently returns a vector. Data frames behave the same way with df[, "dex"], while tibbles never drop a dimension.
Data frames
A data frame is a table whose columns can have different types: text, numbers, categories. Sample sheets, results tables and plotting data are all data frames. Use $ to take one column, and [rows, columns] to take part of the table.
Under the surface, a data frame is a list of columns that all have the same length. That is why $ works on both: it takes a named item out of a list, and in a data frame each item is a column. Each row is one observation, such as one sample, and each column is one variable, such as the donor or the treatment. Inside square brackets, the part before the comma chooses rows and the part after it chooses columns; leaving one part empty means "all of them".
A few patterns cover most of what you will need. samples[1:2, ] gives the first two rows and every column. samples[, c("sample", "dex")] gives every row but only two columns, chosen by name. samples[samples$dex == "trt", ] uses logical indexing from Vectors, functions and loops to keep only the treated rows: the test produces one TRUE or FALSE per row, and the rows marked TRUE are kept. nrow() and ncol() report the size, and head() prints the first six rows, which is safer than printing a table with thousands of lines.
Choose columns by name, not by number, wherever you can. samples[, 3] works today, but if someone adds a column to the sample sheet next month, column 3 becomes something else and your code carries on without complaint. samples$dex still means the treatment column whatever the order. Rows are a different matter: their order rarely means anything, so you pick them with a condition, as above, rather than by position.
Read the course sample sheet
Save airway_samples.csv from the course files into your project folder, then:
A CSV file (comma-separated values) is a plain text table: one line per row, with commas between the columns, and the first line holding the column names. Spreadsheet programs can open and save it, and every analysis language can read it.
read.csv() looks for the file in the working directory, which is your project folder if you opened the RStudio Project from Setting up R and RStudio. The most common first error is "cannot open the connection", with a warning that says "No such file or directory". It almost always means the file is somewhere else, often still in Downloads, or that the name differs slightly, for example a hidden extra .csv added by the operating system. The Files pane shows exactly what is in the folder.
A second pitfall comes from spreadsheet settings. In countries that write decimals with a comma, spreadsheet programs often save "CSV" files with semicolons between columns. read.csv() then reads each line as one long text column, and str() shows a single variable instead of three. read.csv2() reads that semicolon form. This is one more reason to look at str() straight after reading.
samples <- read.csv("airway_samples.csv")
str(samples)
table(samples$cell, samples$dex)What this does
read.csv("airway_samples.csv")reads the file from the project folder and returns a data frame, stored assamples.str(samples)shows the structure of the object: its kind, its size, and the type and first values of every column.table(samples$cell, samples$dex)counts how many rows have each combination of donor and treatment. With two columns as input it gives a cross-tabulation, rows by columns.
Reading the output. The first line says samples is a data frame of 8 observations (rows) and 3 variables (columns). Each following line starts with $ and a column name, then its type, chr for character (text), then its first few values; the ... means more follow. In the table, the rows are the four donors and the columns the two treatments, and every cell is 1. R sorts text alphabetically, which is why trt comes before untrt, and why N61311 is last (the character 6 sorts after 0). That alphabetical order is often not the order you want, which is what factors fix below.
Notice that all three columns are chr. That is right for the sample codes and the donors, which are labels, not quantities. If a column you expect to hold numbers shows up as chr, stop there, because the file contains something that is not a number. The cross-table is a check on the design as much as on the file: a 0 or a 2 in any cell would mean a sample is missing or duplicated, and you would want to know that before any analysis.
These are the eight samples of the airway experiment, a published RNA-seq study analysed in full in Module 4: RNA-seq analysis: airway smooth muscle cells from four donors (cell), each either treated with the steroid dexamethasone (trt) or left untreated (untrt). The table confirms the design: every donor has exactly one sample of each. That is a paired design, and it will matter in Comparing two groups.
Dexamethasone is a synthetic glucocorticoid, an anti-inflammatory steroid of the kind used to treat asthma, so the experiment asks which genes airway muscle cells switch on or off in response to it. The sample codes beginning with SRR are run identifiers from the Sequence Read Archive, the public database where the raw reads are stored.
Why does pairing help? Cells from different people differ in many genes for reasons that have nothing to do with the drug. When each donor contributes one treated and one untreated sample, you can compare the two samples from the same donor, so those personal differences cancel out and the drug's effect is easier to see.
The layout described in this lesson, one row per observation and one column per variable, is called tidy data. It was set out by Hadley Wickham in "Tidy data" (Journal of Statistical Software, 2014), which also explains why messy layouts make analysis harder. The airway experiment is published as Himes et al. 2014, PLOS ONE, a study of how airway smooth muscle cells respond to glucocorticoids. Cite that paper when you use the data, and the data package itself with citation("airway"). Tidy is a convention, not a law: count matrices, with genes in rows and samples in columns, are deliberately wide, because the statistics need them that way.
Factors
A factor stores categories with a fixed order. The first level is the reference that other groups are compared with. Untreated should be the reference:
A level is one of the allowed categories. Choosing the reference matters because results are reported as "this group compared with the reference". With untreated as the reference, a gene that rises with dexamethasone shows a positive change; with treated as the reference, the same gene would show a negative one. The biology is identical, but the sign flips, and reports are often misread.
samples$dex <- factor(samples$dex, levels = c("untrt", "trt"))
levels(samples$dex)
write.csv(samples, "samples_checked.csv", row.names = FALSE)What this does
factor(samples$dex, levels = c("untrt", "trt"))turns the text column into a factor whose levels are in the order you give, untreated first. The result replaces the old column.levels(samples$dex)prints the levels in order, so you can check the reference.write.csv(…, "samples_checked.csv", row.names = FALSE)saves the table as a new CSV file.row.names = FALSEstops R adding an extra first column of row numbers (1 to 8), which would otherwise appear as an unnamed column next time you read the file.
Reading the output. [1] "untrt" "trt" shows the two levels with untreated first, so it is now the reference. One warning: any value that is not in levels silently becomes NA. If one row had said Trt with a capital T, it would turn into a missing value without an error. A table() after making a factor catches this; Cleaning data with dplyr and tidyr deals with exactly that kind of mess.
Factor order also controls how results are drawn. In Figures with ggplot2, ggplot2 places the groups along an axis, and in the legend, in level order; without the factor, untreated and treated would appear alphabetically, treated first. Setting the levels once, near the top of the script, keeps tables, models and figures consistent.
A factor's levels live only inside R. A CSV file is plain text, so samples_checked.csv stores the words "untrt" and "trt" but not their order. When you read the file back in, dex is text again and you must run the factor() line again. Two related functions are useful. relevel(samples$dex, ref = "untrt") moves one level to the front without retyping the others. droplevels() removes levels that no longer have any rows, for example after you filter out every treated sample, so that tables and plots do not show an empty group.
Key point
Run str() and a quick table() every time you load data. It takes seconds and catches most problems before they reach your results.
For example, if a column of read counts appears in str() as chr instead of int or num, some cell in the file contains text, perhaps "n/a" or a stray space, and every calculation on that column will fail or mislead until you find it.
Further reading · Module 1R for Data Science, 2nd edition: the free book that covers this module and much more. · All reading for this course
Check your understanding
1. Which structure suits a count table of numbers, genes by samples?
2. What does the airway design table show?
3. Why set the factor levels to c("untrt", "trt") ?
Up next
Lesson 4
Cleaning data with dplyr and tidyr