Skip to content

BEP 4: Test Framework

Fabio Pliger edited this page Jul 30, 2015 · 1 revision
BEP 4 PyTest Test Framework
Authors Fabio Pliger
Status WIP
Discussion https://github.com/ContinuumIO/bokeh/issues/1984

The aim of this BEP is to improve our test infrastructure by migrating from unittest/nose to pytest.

Motivation

Pytest offers many nice features that would help to improve our tests code base quality, maintainability and flexibility. Here's a short list of the advantages that pytest would bring:

  1. clean and intuitive syntax: would definitely help with tests readability and maintainability. A lot of the unittest boilerplate code could just go away. Test cases can be written as much simpler class or just a collection of shorter functions.
  2. support unittest tests syntax/style: would make the transition from unittest easier as we can have a mixed period of unittest and new pytest tests working without old tests breaking.
  3. better tests patterns: it provides intuitive and elegant assertions and raises context managers that makes testing exceptions better
  4. fixtures: pytest fixtures are very powerful, reusable and flexible, more info here: https://pytest.org/latest/fixture.html
  5. useful builtin fixtures: monkeypatch, tmpdirand other builtin fixtures are extremely powerful!
  6. powerful plugins: pytest provides a powerful plugins system that can be used to extend the default features
  7. pdb debugging & friends: --pdb argument makes local debugging waaaaaay easier by automatically opening a pdb session when tests fails. Also other cmd args to enable/disable tests, change captured output makes life quite easier.
  8. configuration files: pytest configuration files (ini, cfg, conftest) let users change tests behaviour very easily

Drawbacks

  1. current unittest codebase: our current test base is entirely based on unittest. The migration to pytest will not break current tests but the mixed codebase will be a bit annoying for a limited period of time.