Testing

Michael C Sachs

2025-05-27

Bugs!

1. Crash: function call exits with error

2. Incorrect behavior

3. Unexpected behavior

library(MASS)
data(Pima.te)

## Crash
subset( head(Pima.te), npreg > a )

Error in eval(e, x, parent.frame()): object 'a' not found

## Incorrect behavior
subset( head(Pima.te), type = "yes" & npreg > 2 )

  npreg glu bp skin  bmi   ped age type
1     6 148 72   35 33.6 0.627  50  Yes
2     1  85 66   29 26.6 0.351  31   No
3     1  89 66   23 28.1 0.167  21   No
4     3  78 50   32 31.0 0.248  26  Yes
5     2 197 70   45 30.5 0.158  53  Yes
6     5 166 72   19 25.8 0.587  51  Yes

## Unexpected behavior
subset( head(Pima.te), type == "yes" & npreg > 2 )

[1] npreg glu   bp    skin  bmi   ped   age   type 
<0 rows> (or 0-length row.names)

Tools

traceback to find where error occurred:

library(MASS)
data(Pima.te)
subset(Pima.te, npreg > a)
traceback()

debug to execute code line by line:

• debug(<function-name>) to start debugging

• At debug-prompt:

  • n or Enter to execute current line, continue to next
  • c to run function from the current line to end
  • Q to quit
  • any variable, expression is evaluated in the current environment

• undebug(<function-name>) to restore normal behavior

browser to set stop points when running code:

• Add browser() command to code at strategic locations
• Function will execute to browser and open debugger as above

Rstudio Debug menu: similar fucntionality, but can only be started for R scripts

Example

f1 = function(x, y, flag = TRUE, n = 1000) 
{
  if ( missing(y) ) 
  {
    if (flag | (x > 0)) y = abs(x)
  }
  g = function(z) z + x
  acc = 0
  ## What happened to ?cumsum 
  for (i in 1:1000) acc = acc + runif(1) 
  g(y)
}

Run tests:

f1(1)
f1(-1)
f1(1, flag = FALSE)
f1(-1, flag = FALSE)

1. Run traceback (not very useful)

2. Use debug to run the critical case line-by-line as far as practical

3. Note how the focus jumps between editor and console; editor has some decent Debug menu items, but inspection of local vars (e.g. ls()) is in the console

4. Add a browser() statement that allows you to pass over the loop (or use the menu item)

5. Fix the function, remove the browser if necessary & undebug

Minimal replicable example

Simplify any example that generates an error

Smallest self-contained set of data & code that generates the error reliably

• Often finds the problem
• Easier to read & understand on Stackoverflow etc.
• Shows respect for other people’s time

Automated testing at build

Up until now, your workflow probably looks like this:

1. Write a function.
2. Load it with or devtools::load_all().
3. Experiment with it in the console to see if it works.
4. Go back to 1.

While this is testing, sometimes a more formal approach is better.

Let’s start by setting up our package to use the testthat: usethis::use_testthat().

Unit tests

Advantages:

1. Better code, you are explicit about how the code behaves
2. Better structure, many small components working together
3. Easier to start up, all your tests are documented in a script
4. Fearless development, don’t have to worry about unknowingly breaking something

Unit tests with testthat are defined in terms of expectations. The package provides a suite for functions that have the expect_ prefix that compare the values of different objects:

library(testthat)
expect_equal(1, 1 + 1)

Error: 1 not equal to 1 + 1.
1/1 mismatches
[1] 1 - 2 == -1

expect_lt(1, 1 + 1)

General principles:

1. When you are tempted to write a “print” statement, write a test instead.
2. Write tests for complex code with lots of interdependencies
3. Write a test when you discover a bug.

Example

library(palmerpenguins)

idex <- vector(mode = "list", length = length(unique(penguins$species)))
names(idex) <- unique(penguins$species)
for(sp in unique(penguins$species)) {
    subs <- subset(penguins, species == sp & body_mass_g > 4750)
    idex[[sp]] <- sample(subs$body_mass_g, replace = TRUE)
}

• What bug did we discover in the sampling function?
• What is the expected behavior in this case and how does it fail?
• Write a test!

Organizing tests

Alternatives

• You don’t have to use testthat

devtools::check()

• CRAN automatically runs the code in the tests subdirectory
• If they fail, you will be notified.
• CRAN does other things as part of check! Let’s review them.