Image Comparison Mode#

This how-to guide will show you how to use the image comparison mode of pytest-mpl. This is the default mode when pytest-mpl is invoked with --mpl.

In this mode, pytest-mpl will compare the images generated by the test with a baseline image. If the images are different, to a specified RMS tolerance, the test will fail. The test will also fail if the baseline image is not found or if the test does not generate a figure. If the figure has a different aspect ratio to the baseline image, the test will also fail.

Generating baseline images#

Once a suite of image comparison tests have been written, baseline images should be generated by setting --mpl-generate-path:

pytest --mpl-generate-path=your_project/tests/baseline

You should choose a directory within you repository to store the baseline images. The generated images should be checked to ensure they are as expected. The images should then be committed to the repository.

To assist with inspecting the generated images, a HTML summary report can be generated by setting --mpl-generate-summary:

pytest \
  --mpl-generate-path=your_project/tests/baseline \
  --mpl-generate-summary=html,json

HTML JSON

Running image comparison tests#

When running the tests, the --mpl flag should be used to enable baseline image comparison testing:

pytest --mpl \
  --mpl-baseline-path=your_project/tests/baseline

The baseline path should be set to the directory containing the baseline images. If the baseline images are not found, the test will fail.

Optionally, a HTML summary report can be generated by setting --mpl-generate-summary:

pytest --mpl \
  --mpl-baseline-path=your_project/tests/baseline \
  --mpl-results-path=results \
  --mpl-generate-summary=html,json

HTML JSON

The --mpl-results-path flag can be used to set the directory where the generated HTML summary will be stored. If this is not set, the images will be stored in a temporary directory.

Continue reading#

pytest-mpl has many configuration options that can be used to customize the behavior of the image comparison mode. Only a few of the most commonly used options are covered in this guide. See the configuration options documentation for full details.