test_suite.qmd

# Test suite {#sec-tests-suite}

<!---
https://jakubsob.github.io/blog/

https://www.freecodecamp.org/news/clean-coding-for-beginners/
-->

```{r}
#| eval: true 
#| echo: false 
#| include: false
source("_common.R")
library(testthat)
library(withr)
library(logger)
```


```{r}
#| label: co_box_tldr
#| echo: false
#| results: asis
#| eval: true
co_box(
  color = "b", 
  look = "default", hsize = "1.10", size = "1.05",
  header = "TLDR &emsp; ![](images/testthat.png){width='8%'}",
  fold = TRUE,
  contents = "
  
<br>

**`testthat` Workflow:**
  
- `use_testthat()`: setup testing infrastructure in your app-package\n
  
  - Include edition (i.e., `use_testthat(3)`)\n
  
- `use_test()`: creates new test files (with `test-` prefix)\n
  
  - Each file under `R/` should a corresponding `test-` file\n
  
- `test_active_file()`: runs tests in the current open test file\n
  
- `test_coverage_active_file()`: test coverage for the current open test file\n
  
**Behavior-driven development functions:**
  
- `describe()`: provides context (user specification or feature) for tests and test code\n

- `it()`: used to test functional requirement (i.e., expectation functions).\n
  
  "
)
```

---

Testing Shiny applications poses some unique challenges. Shiny functions are written in the context of its reactive model,[^tests-shiny-reactive] so some standard testing techniques and methods for regular R packages don't directly apply. This chapter covers setting up the `testthat`’s infrastructure, keyboard shortcuts for commonly used functions, and running tests in RStudio ![](images/rstudio-icon.png){height=20} vs. Positron ![](images/positron.png){height=20}.

[^tests-shiny-reactive]: The ['Reactivity - An overview'](https://shiny.posit.co/r/articles/build/reactivity-overview/) article gives an excellent description (and mental module) of reactive programming.

```{r}
#| label: shinypak_apps
#| echo: false
#| results: asis
#| eval: true
shinypak_apps(regex = "14", branch = "14_tests-suite")
```

## [`testthat`]{style="font-size: 1.05em;"} framework

`testthat` is the standard package for testing in R packages and one of the most widely used and supported packages on CRAN. Its widespread adoption is due to its ability to simplify the setup, creation, and execution of unit tests.

In our app-package, we'll use `testthat` unit tests to ensure the underlying logic (i.e., non-reactive utility functions) behaves correctly. We can extend `testthat`'s framework for integration tests with Shiny's [`testServer()`](https://shiny.posit.co/r/articles/improve/server-function-testing/) function and system tests with the [`shinytest2` package](https://rstudio.github.io/shinytest2/). Together, these tools provide a comprehensive testing suite for an app-package.


## [Setting up `testthat` tests]{style="font-size: 0.95em;"} {#sec-tests-suite-use-testthat}

The `testthat` package has been around for over a decade and thus has undergone various changes that require us to specify the edition we intend to use (currently, it's the third):[^tests-testthat-edition]

[^tests-testthat-edition]: Read more about changes to the third edition to `testthat` in [R Packages, 2ed](https://r-pkgs.org/testing-basics.html#introducing-testthat)

```{r}
#| eval: false 
#| code-fold: false
usethis::use_testthat(3)
```

Setting up your testing infrastructure with `use_testthat()` does the following:

1.  In the `DESCRIPTION` file, `testthat (>= 3.0.0)` is listed under `Suggests`

2.  `Config/testthat/edition: 3` is also listed in the  `DESCRIPTION` to specify the `testthat` edition
    
3.  A new `tests/` folder is created, with a `testthat/` subfolder

4.  The `tests/testthat/testthat.R` file is created 

We now have a `tests/` folder to store our `testthat` tests.

```{bash}
#| eval: false
#| code-fold: false
tests/
  ├── testthat/
  └── testthat.R #<1>

2 directories, 1 file
```
1. Referred to as the 'test runner,' because it runs all our tests (do not edit this file).

## Creating unit tests {#sec-tests-suite-use-test}

```{r}
#| label: git_box_10_test-suite
#| echo: false
#| results: asis
#| eval: true
git_margin_box(
  contents = "launch",
  fig_pw = '65%', 
  branch = "10_test-suite", 
  repo = 'sap')
```

The standard workflow for writing `testthat` unit tests consists of the following:

**New tests** are created with `usethis::use_test()`:

```{r}
#| eval: false
#| code-fold: false
usethis::use_test("scatter_plot") 
```
  -   `testthat` recommends having a corresponding test file in `tests/testthat/` (with the `test-` prefix) for the files in `R/`.

### [`test-`]{style="font-size: 0.95em;"} files

**Test files**: the IDE will automatically create and open the new test file: 

```{verbatim}
#| eval: false
#| code-fold: false
✔ Writing 'tests/testthat/test-scatter_plot.R'
• Modify 'tests/testthat/test-scatter_plot.R'
```

### [`test_that()`]{style="font-size: 0.95em;"} tests {#sec-tests-suite-test-that}

Each new test file contains a boilerplate `test_that()` **test**: 

```{r}
#| eval: false
#| code-fold: false 
test_that(desc = "multiplication works", code = { # <1>
 
})
```
1. `desc` is the test context (supplied in `"quotes"`), and `code` is the test code (supplied in `{curly brackets}`).

### [`expect_`]{style="font-size: 0.95em;"}ations {#sec-tests-suite-expectations}

The **expectations** typically have two arguments: `observed` and `expected`.

```{r}
#| eval: false
#| code-fold: false 
#| collapse: true
expect_equal( # <1> 
  object = 2 * 2, # <2> 
  expected = 4 # <3> 
  ) 
```
1. A `testthat` expectation function  
2. The output or behavior being tested  
3. A predefined output or behavior    

The `observed` object is an artifact of some code we've written, and it's being compared against an `expected` result.
  
### BDD test functions {#sec-tests-suite-bdd-intro}

```{r}
#| label: co_box_positron_version
#| echo: false
#| results: asis
#| eval: true
co_box(
  color = "b", 
  look = "simple", 
  hsize = "1.10", 
  size = "1.05",
  header = "![](images/positron.png){height=20} Positron Version", 
  fold = TRUE,
  contents = "At the time of this writing, the [2024.09.0-1](https://github.com/posit-dev/positron/releases/tag/2024.09.0-1) pre-release of Positron was available for testing."
)
```

`testthat` also has two behavior-driven development (BDD) functions for performing tests: `describe()` and `it()`. 

> "*Use `describe()` to verify that you implement the right things and use [`it()`] to ensure you do the things right.*" - `testthat` [documentation](https://testthat.r-lib.org/reference/describe.html)

```{r}
#| eval: false
#| code-fold: false 
describe("Description of feature or specification", # <1> 
      code = { 
    it("Functionality under test", # <2> 
        code = { # <3> 
            expect_equal( 
            object = 2 * 2, 
            expected = 4 
            ) # <3> 
        }) # <2> 
    }) # <1> 
```
1. `describe()` the feature or specification   
2. Capture `it()` in a test   
3. Write expectations   

We'll cover BDD more in the [next chapter](test_specs.qmd), but for now just know that each call to `it()` behaves like `test_that()`.

## Running tests {#sec-tests-suite-running-tests}

Another [`devtools`](development.qmd) habit to adopt is regularly **writing and running tests**. Below we'll cover writing and running tests in RStudio ![](images/rstudio-icon.png){height=20} and Positron ![](images/positron.png){height=20}.


### Keyboard shortcuts {#sec-tests-suite-keyboard-shortcuts}

R Packages, 2ed also [suggests](https://r-pkgs.org/testing-basics.html#run-tests) binding `test_active_file()` and `test_coverage_active_file()` to keyboard shortcuts. I **highly** recommend using a shortcut while developing tests because it will improve your ability to iterate quickly.

::: {layout="[54, -1, 45]" layout-valign="top"}

[`devtools` function]{style="font-weight: bold; font-size: 1.20em"}

[Keyboard shortcut]{style="font-weight: bold; font-size: 1.20em"} 

:::

::: {layout="[54, -1, 45]" layout-valign="bottom"}

[`test()`]{style="font-weight: bold; font-size: 0.95em"}

[<kbd>Ctrl/Cmd</kbd> + <kbd>Shift</kbd> + <kbd>T</kbd>]{style="font-weight: bold; font-size: 0.80em"}

:::

::: {layout="[54, -1, 45]" layout-valign="bottom"}

[`test_active_file()`]{style="font-weight: bold; font-size: 0.95em"}

[<kbd>Ctrl/Cmd</kbd> + <kbd>T</kbd>]{style="font-weight: bold; font-size: 0.80em"} 

:::

::: {layout="[54, -1, 45]" layout-valign="bottom"}

[`test_coverage_active_file()`]{style="font-weight: bold; font-size: 0.95em"}

[<kbd>Ctrl/Cmd</kbd> + <kbd>Shift</kbd> + <kbd>R</kbd>]{style="font-weight: bold; font-size: 0.80em"} 

:::

<br>

Follow [these instructions](https://docs.posit.co/ide/user/ide/guide/productivity/custom-shortcuts.html) to create a new keyboard shortcut in RStudio ![](images/rstudio-icon.png){height=20}. Positron ![](images/positron.png){height=20} already includes the `devtools::test()` shortcut, but the other two (`test_active_file()` and `test_coverage_active_file()`) will have to be added manually (see instructions [here](https://github.com/posit-dev/positron/wiki/Keyboard-Shortcuts)).

### Tests in RStudio ![](images/rstudio-icon.png){height=20}



In RStudio ![](images/rstudio-icon.png){height=20}, `test_active_file()` (or [<kbd>Ctrl/Cmd</kbd> + <kbd>T</kbd>]{style="font-weight: bold; font-size: 0.90em"}) will test the current test file: 

![`test_active_file()` or <kbd>Ctrl/Cmd</kbd> + <kbd>T</kbd> in RStudio](images/test_suite_rstudio_devtools_test_active_file.png)

The output will provide feedback on whether the test passes or fails (and occasionally some encouragement).

When we've written multiple test files, we can run all the tests in our app-package using the **Build** pane or the keyboard shortcut ([<kbd>Ctrl/Cmd</kbd> + <kbd>Shift</kbd> + <kbd>T</kbd>]{style="font-weight: bold; font-size: 0.90em"})

::: {layout="[50, 50]" layout-valign="top"}

![Run all tests](images/tests_build_pane_test.png){width='100%'}

![`devtools::test()`](images/test_suite_rstudio_devtools_test.png)

:::

RStudio ![](images/rstudio-icon.png){height=20} can be configured to include additional columns, which can be helpful during test development. Below is an example workflow with the `R/scatter_plot.R` file, it's accompanying test file, the **Build** pane, and the **Console.**

![Test workflow in RStudio](images/test_suite_rstudio_test_workflow.png){width='100%' fig-align='center'}

### Tests in Positron ![](images/positron.png){height=20}

If we click on the **Testing** sidebar menu item with our test file open, Positron ![](images/positron.png){height=20} will display the hierarchy of the `describe()` and `it()` functions:

::: {.column-margin}

![Testing icon in Positron sidebar menu](images/test_suite_positron_testing_icon.png){width=50} 

:::

![BDD testing functions in Positron](images/test_tools_positron_bdd_test.png){width='100%' fig-align='center'}

Positron ![](images/positron.png){height=20} gives us multiple options for running tests. For running individual tests, we can use the **Run Test** icons:

::: {layout="[50, 50]" layout-valign="bottom"}

![**Run Test** in **Testing** sidebar menu item](images/test_suite_positron_run_test.png)

![**Run Test** in test file](images/test_suite_positron_run_test_file.png)

:::

When testing the active file in Positron ![](images/positron.png){height=20}, the results are displayed in the **Console**:

![`test_active_file()` in Positron](images/test_suite_positron_devtools_test_active_file.png){width='80%' fig-align='center'}

To run all the tests in our app-package, we can use the keyboard shortcut or the **Run Tests** icon:

::: {layout="[50, 50]" layout-valign="bottom"}

![**Run Tests** in **Testing** sidebar menu item](images/test_suite_positron_run_tests.png)

![**Run Tests** in test file](images/test_suite_positron_run_tests_file.png)
:::

The [<kbd>Ctrl/Cmd</kbd> + <kbd>Shift</kbd> + <kbd>T</kbd>]{style="font-weight: bold; font-size: 0.90em"} shortcut will call `devtools::test()` and display the results in a new R **Terminal** task:

![](images/test_suite_positron_devtools_test.png){width='100%'}


<!--
:::: {.panel-tabset}

### RStudio ![](images/rstudio-icon.png){height=20}

### Positron ![](images/positron.png){height=20}

::::
-->


## Recap {.unnumbered}

```{r}
#| label: co_box_recap
#| echo: false
#| results: asis
#| eval: true
co_box(
  color = "g", 
  look = "default", hsize = "1.10", size = "1.05",
  header = "RECAP &emsp; ![](images/testthat.png){width='8%'}",
  fold = FALSE,
  contents = "
  
<br>
  
**`testthat` setup**\n
  
- `use_testthat()`: sets up testing infrastructure in your app-package\n
  
**Test files**\n
  
- `use_test()`: creates new test files (with `test-` prefix). The test file names should *generally* match the file names be below` R/`.\n
  
**BDD test functions**\n
  
- `describe()`: Feature descriptions and any relevant background information\n
- `it()`: Scenarios and test code with expectations (`Then` statement =  functional requirement).\n
  
**Running tests**\n
  
- `test_active_file()`: runs tests in the current open test file\n
  
- `test_coverage_active_file()`: test coverage for the current open test file\n
  
"
)
```


```{r}
#| label: git_contrib_box
#| echo: false
#| results: asis
#| eval: true
git_contrib_box()
```