Testing
Describe testing categories and tools.
Python test scripts in the repository tests/
directory are run with
pytest
:
python -m pytest tests/
We use a few pytest plugins. To install them first, it should be sufficient to
use the requirements-testing.txt
file in the root of the repository.
pip install -r requirements-testing.txt
Some test scripts or examples rely on data that lives in a separate
testdata repository.
This repository is noted as a
git submodule
in the .gitmodules
configuration file. GitHub respects and records
the repository link, so the testdata repository will be cloned with
https://github.com/SCALE-MS/scale-ms if you used the --recursive
option
to git clone. Otherwise, you will have to manually initialize
and update the submodule or otherwise acquire the test data.
Unit tests, system tests, and integration tests
The test suite in tests/
includes a lot of integration tests that probe
interactions with full RADICAL Pilot sessions because we have had problems
reproducing viable execution environments and because we use RP features and
interfaces that may not be mature.
Refer to the docker/README.md
in the repository for more about
reproducible testing environments.
We install several pytest markers and command line options for the
tests in tests/
. Refer to tests/conftest.py
or use
pytest tests --help (look for the custom options:
section).
Acceptance tests
The automated GitHub Actions test pipeline includes command line invocations of example scripts. We should continue to run short examples to ensure that the scripting interface behaves as expected.