# Python : Testing : pytest : Overview
\[ [pypi](https://pypi.org/project/pytest/) | [src](https://github.com/pytest-dev/pytest/) | [docs](https://docs.pytest.org/en/7.4.x/contents.html) | [api](https://docs.pytest.org/en/7.4.x/reference/reference.html) ]
> [!NOTE] Bash Autocompletion
> ```bash
> $ pip install argcomplete
> $ register-python-argcomplete pytest >> ~/.bashrc
> ```
## Cheatsheet
**pyproject.toml** (preferred over legacy `pytest.ini`)
```toml
[ tool.pytest.ini_options ] # will change to "tool.pytest" in a future pytest version
addopts = "--capture=no --import-mode=importlib --showlocals --strict-markers -vv"
pythonpath = [ "." ]
testpaths = [ "tests/" ]
```
**conftest.py**
```python
import pytest
@pytest.fixture
def foo():
...
@pytest.fixture( scope="function*|class|module|package|session", autouse=False, name=None )
def bar():
...setup...
yield ...
...cleanup...
```
**Test File**
```python
import pytest
pytestmark = [ pytest.mark.foo, ... ] # applied to entire module
@ptest.mark.foo # applied to function
def test_func( ...fixtures... ):
assert ...
@pytest.mark.foo # applied to every method in class
class TestClass:
# a new instance of the class is created for each test method
@pytest.mark.foo # applied to method
def test__method( self, ...fixtures... ):
assert ...
# assert raised exceptions - match uses re.search()
with pytest.raises( <ExceptionClass>[, match="pattern"] )[ as excinfo]:
...
# excinfo is an ExceptionInfo instance - a wrapper around the actual exception
excinfo.type
excinfo.value
excinfo.traceback
```
**CLI Invocation**
```bash
$ pytest --fixtures # show available fixtures
$ pytest --markers # show available markers
$ pytest [options] [<arg1> ...] # arg -> file, dir, module, dotted-module-path
# startup options
--rootdir DIR # set rootdir
-c | --config-file FILE # set configfile
# arg patterns
pytest path/
pytest [path/to/]module.py
pytest [path/to/]module.py::test_func
pytest [path/to/]module.py::TestClass[::test_method]
pytest package
pytest [package.]module
pytest [package.]module::test_func
pytest [package.]module::TestClass[::test_method]
# test discovery / collection options
--import-mode MODE # defaults to 'prepend'; 'importlib strongly recommended
--pyargs # force interpretation of args as python packages
-m <mark-exp> # -m foo | -m not foo | -m foo and not bar
-k <keyword-exp> # only run tests that match the Python-evaluable case-insensitive expression
# where names are substring-matched against test function and class names
# ex: -k "k1 and not k2"
```
```
-o name=value # override config option from CLI
addopts = "..." # add to CLI options from config (pyproject.toml)
```
## Execution Details
#### Invocation
Three ways to invoke:
```bash
$ pytest
$ python -m pytest
```
```python
exit_code = pytest.main( args=None, plugins=None ) # args & plugins are lists of strings
# reads sys.argv if no args
```
#### Startup
1) Determine `rootdir`....
- from `--rootdir` option if provided
- else if args -> set to common ancestor of all args
- else CWD
2) Determine `configfile`...
- from `-c | --config-file` option if provided
- else recursively search `rootdir` for `pyproject.toml` with `tool.pytest.ini_options` section
3) Begin Test Discovery.
> [!NOTE]
> The determined `rootdir` and `configfile` are printed as part of the header, and accessible via the `pytestconfig` fixture.
> [!WARNING]
> `rootdir` is NOT used to modify `sys.path`. For that, use `PYTHONPATH` env var or pytest config option`pythonpath`.
#### Test Discovery & Collection
1) Start from...
- args if specified
- else config option `testpaths` if set
- else CWD
2) Recurse into directories unless they match config option `norecursedirs`.
3) In those directories, import modules matching patterns in config option `python_files` (default: `"test_*.py *_test.py"`).
4) From those modules, collect...
- test functions that match config option `python_functions` (default: `"test*"`)
- test methods that match config option `python_functions` within test classes that match `python_classes` (default: `"Test*"`)
- test classes must not contain a `__init__`
- class and static methods are also considered
> [!NOTE] conftest.py
> The `conftest.py` file in `rootdir` will be loaded before Test Discovery begins. Any `conftest.py` files in subdirectories will be discovered and loaded *during* Test Discovery.
## Fixtures
- Fixtures are created once per `scope`, which defaults to "function".
- Fixtures are either...
- requested explicitly (via function args)
- applied to function / class / module by marker
- applied to entire test suite via `usefixtures` config option
- applied to entire test suite via `autouse=True`
- fixtures defined in a non-root `conftest.py` will only be applied to that subtree of tests
- Fixtures can request fixtures.
- You can override a fixture by defining a new one with the same name more locally, either in `subdir/conftest.py` or directly in the test module,
#### Requesting / Applying
To request a fixture in a test function, add a function arg by the same name:
```python
def test_foo( tmp_path ): # request the `tmp_path` fixture
...
```
Or apply fixtures using marks:
```python
pytestmark = [ usefixtures( "foo", ... ) ] # all tests in module
@pytest.mark.usefixtures( "foo", ... ) # single test function
def test_func():
...
@pytest.mark.usefixtures( "foo", ... ) # all methods in test class
class TestClass:
...
```
Or apply fixtures to all tests in the entire suite using `pyproject.toml`:
```toml
usefixtures = "foo ..."
```
Or apply to all tests in the fixture definition:
```python
@pytest.fixture( autouse=True, scope=... )
def my_fixtures():
...
```
#### Built-in Fixtures
- capfd -> Capture, as text, output to file descriptors 1 and 2.
- capsys -> Capture, as text, output to `sys.stdout` and `sys.stderr`.
- caplog -> Control logging and access log entries.
- monkeypatch -> Temporarily modify classes, functions, dictionaries, `os.environ`, and other objects.
- tmp_path -> Provide a `pathlib.Path` object to a temporary directory which is unique to each test function.
**APIs**
```python
# output capturing
capfd.readouterr() # namedtuple( out, err )
caplog.messages # list of format-interpolated log messages
caplog.text # string containing formatted log output
caplog.records # list of logging.LogRecord instances
caplog.record_tuples # list of (logger_name, level, message) tuples
caplog.clear() # clear captured records and formatted log output string
# monkeypatching
monkeypatch.setattr( obj, name, value, raising=True )
monkeypatch.delattr( obj, name, raising=True )
monkeypatch.setitem( mapping, name, value )
monkeypatch.delitem( obj, name, raising=True )
monkeypatch.setenv( name, value, prepend=None )
monkeypatch.delenv( name, raising=True )
monkeypatch.syspath_prepend( path )
monkeypatch.chdir( path )
monkeypatch.context()
```
#### Custom
> [!Tip] Dynamic Scope
> Pass callable instead of string. (https://docs.pytest.org/en/latest/how-to/fixtures.html#dynamic-scope)
> [!NOTE] Parameterizing Fixtures
> Will cause every test using that fixture to be run once for every value in the set.
> [!TIP] Passing Data to Fixtures
> ```python
> @pytest.fixture
> def foo( request ):
> if marker := request.node.get_closest_marker( "foo_data" ):
> return marker.args[ 0 ]
>
> @pytest.mark.foo_data( 42 )
> def test_func( foo ):
> assert foo == 42
> ```
## Misc
#### Exit Codes
```
pytest.ExitCode (enum)
0 -> All tests were collected and passed successfully
1 -> were collected and run but some of the tests failed
2 -> Test execution was interrupted by the user
3 -> Internal error happened while executing tests
4 -> pytest command line usage error
5 -> No tests were collected
```
#### Regarding sys.path
```
Scenario 1: python -m pytest
Scenario 2: pytest
Scenario 3: pytest + config:pythonpath="foo"
1 2 3 sys.path
─── ─── ─── ───────────────────────────────────────────────────────────────────────────────────────────
+ ~/src
+ ~/src/foo
+ + ~/.cache/pypoetry/virtualenvs/foo-backend-dfiEBeF_-py3.12/bin
+ + + ~/.pyenv/versions/3.12.0/lib/python312.zip
+ + + ~/.pyenv/versions/3.12.0/lib/python3.12
+ + + ~/.pyenv/versions/3.12.0/lib/python3.12/lib-dynload
+ + + ~/.cache/pypoetry/virtualenvs/foo-backend-dfiEBeF_-py3.12/lib/python3.12/site-packages
```