Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
name: Release

on:
push:
tags: ['[0-9]+.[0-9]+.[0-9]+*']

permissions:
contents: read

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Check the tag is on the default branch
env:
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
run: |
if ! git merge-base --is-ancestor "$GITHUB_SHA" "origin/$DEFAULT_BRANCH"; then
echo "::error::Tag $GITHUB_REF_NAME points at a commit that is not on $DEFAULT_BRANCH"
exit 1
fi
- uses: actions/setup-python@v7
with:
python-version: '3.14'
- name: Check the tag matches the package version
run: |
version=$(python -c 'import tomllib; print(tomllib.load(open("pyproject.toml", "rb"))["project"]["version"])')
if [ "$version" != "$GITHUB_REF_NAME" ]; then
echo "::error::Tag $GITHUB_REF_NAME does not match version $version in pyproject.toml"
exit 1
fi
- name: Test
run: |
pip install '.[playwright]' -r requirements.txt
python -m playwright install --with-deps chromium
python -m pytest tests
- name: Build
run: pip install build && python -m build
- uses: actions/upload-artifact@v7
with:
name: dist
path: dist/

publish:
needs: build
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/project/visual-regression-tracker/${{ github.ref_name }}/
permissions:
id-token: write
steps:
- uses: actions/download-artifact@v8
with:
name: dist
path: dist/
- uses: pypa/gh-action-pypi-publish@release/v1
6 changes: 3 additions & 3 deletions .github/workflows/workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ['3.9', '3.10', '3.11', '3.12', '3.13', '3.14']
python-version: ['3.10', '3.11', '3.12', '3.13', '3.14']

steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
uses: actions/setup-python@v7
with:
python-version: ${{ matrix.python-version }}
- name: Install Tox
Expand Down
64 changes: 49 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,15 @@
[![Codacy Badge](https://app.codacy.com/project/badge/Grade/0b98190cea064d6f9f1210da653ba37b)](https://www.codacy.com/gh/Visual-Regression-Tracker/sdk-python?utm_source=github.com&utm_medium=referral&utm_content=Visual-Regression-Tracker/sdk-python&utm_campaign=Badge_Grade)
[![Codacy Badge](https://app.codacy.com/project/badge/Coverage/0b98190cea064d6f9f1210da653ba37b)](https://www.codacy.com/gh/Visual-Regression-Tracker/sdk-python?utm_source=github.com&utm_medium=referral&utm_content=Visual-Regression-Tracker/sdk-python&utm_campaign=Badge_Coverage)

Requires Python 3.10 or later. Image URLs in results need VRT server 5.0.10 or later; use 5.x of this SDK with VRT 6.0.

## Install

```python
```sh
pip install visual-regression-tracker

# or, with playwright integration
pip install visual-regression-tracker[playwright]
pip install "visual-regression-tracker[playwright]"
python -m playwright install
```

Expand All @@ -18,7 +20,7 @@ python -m playwright install
### Import

```python
from visual_regression_tracker import VisualRegressionTracker, Config, TestRun
from visual_regression_tracker import VisualRegressionTracker, Config, TestRun, IgnoreArea
```

### Configure connection
Expand All @@ -43,6 +45,11 @@ config = Config(

# enableSoftAssert - Log errors instead of exceptions
enableSoftAssert=False,

# httpTimeoutInSeconds - How long to wait for the server's answer, 0 for no limit.
# The server compares images while answering track(); with VLM checks or a busy
# comparison queue that can take minutes, so keep it above 300.
httpTimeoutInSeconds=360,
)

vrt = VisualRegressionTracker(config)
Expand All @@ -56,9 +63,12 @@ vrt = VisualRegressionTracker(config)
"apiKey":"tXZVHX0EA4YQM1MGDD",
"ciBuildId":"commit_sha",
"branchName":"develop",
"enableSoftAssert":false
"enableSoftAssert":false,
"httpTimeoutInSeconds":360
}
```

`vrt.json` is read from the current working directory.
```python
vrt = VisualRegressionTracker()
```
Expand All @@ -71,8 +81,10 @@ VRT_APIKEY="tXZVHX0EA4YQM1MGDD" \
VRT_CIBUILDID="commit_sha" \
VRT_BRANCHNAME="develop" \
VRT_ENABLESOFTASSERT=true \
VRT_HTTPTIMEOUTINSECONDS=360 \
python
```
Empty variables are ignored.
```python
vrt = VisualRegressionTracker()
```
Expand All @@ -89,7 +101,7 @@ with vrt:

Without context manager:
```python
vrt.start()
build = vrt.start() # Build(id, projectId)
...
# track test runs
...
Expand All @@ -99,7 +111,7 @@ vrt.stop()
### Assert

```python
vrt.track(TestRun(
result = vrt.track(TestRun(
# Name to be displayed
# Required
name='Image name',
Expand Down Expand Up @@ -145,21 +157,33 @@ vrt.track(TestRun(
))
```

`track` returns a `TestRunResult`: `testRunResponse` (with `status` and the `url` of the test run in the
VRT app) and `imageUrl`, `diffUrl`, `baselineUrl`.

`track` raises `TestRunError` when the status is `new` (no baseline yet) or `unresolved` (difference found);
with `enableSoftAssert` it logs the error instead. `ok`, `approved` and `autoApproved` pass; `failed` passes
too, as in the JavaScript SDK. A status this SDK does not know becomes `TestRunStatus.UNKNOWN`.

An HTTP error from the server raises `ServerError` with `status_code` and the server's `message`.
A request that gets no answer (connection refused, connect timeout after 10 s, read timeout after
`httpTimeoutInSeconds`) raises `RequestError` with the `url`. A read timeout on `track` does not mean
the server dropped the test run: it may still be stored.

### Integration with Microsoft Playwright

#### Imports
```python
from playwright import sync_playwright
from visual_regression_tracker import Config, TestRun
from visual_regression_tracker.p import PlaywrightVisualRegressionTracker
from playwright.sync_api import sync_playwright
from visual_regression_tracker import Config
from visual_regression_tracker.playwright import PlaywrightVisualRegressionTracker
```

#### Start playwright and navigate to page
```python
playwright = sync_playwright().start()
browserType = playwright.chromium
browser = browserType.launch(headless=False)
page = browser.newPage()
page = browser.new_page()
page.goto('https://www.python.org/')
```

Expand Down Expand Up @@ -189,10 +213,10 @@ vrt.stop()

#### Track page
```python
vrt.trackPage(page, imageName[, options])
result = vrt.trackPage(page, imageName[, options])
```

- `page: Page` [Playwright page](https://microsoft.github.io/playwright-python/sync_api.html#playwright.sync_api.Page)
- `page: Page` [Playwright page](https://playwright.dev/python/docs/api/class-page)

- `imageName: str` name for the taken screenshot image

Expand All @@ -206,7 +230,7 @@ vrt.trackPage(page, imageName[, options])
- `height: int` area height in px

- `screenshotOptions: PageScreenshotOptions` configuration for Playwrights `screenshot` method
- `full_page: bool` When true, takes a screenshot of the full scrollable page, instead of the currently visibvle viewport. Defaults to `false`.
- `full_page: bool` When true, takes a screenshot of the full scrollable page, instead of the currently visible viewport. Defaults to `false`.

- `omit_background: bool` Hides default white background and allows capturing screenshots with transparency. Defaults to `false`.

Expand All @@ -226,10 +250,12 @@ vrt.trackPage(page, imageName[, options])

#### Track element handle
```python
vrt.trackElementHandle(elementHandle, imageName[, options])
result = vrt.trackElementHandle(elementHandle, imageName[, options])
```

- `elementHandle: ElementHandle` [Playwright ElementHandle](https://microsoft.github.io/playwright-python/sync_api.html#playwright.sync_api.ElementHandle)
The async API has `trackPageAsync` and `trackElementHandleAsync`. All four return the same `TestRunResult` as `track`.

- `elementHandle: ElementHandle` [Playwright ElementHandle](https://playwright.dev/python/docs/api/class-elementhandle)

- `imageName: str` name for the taken screenshot image

Expand All @@ -252,3 +278,11 @@ vrt.trackElementHandle(elementHandle, imageName[, options])
- `os: str` operating system name, like Windows, Mac, etc.
- `device: str` device name, PC identifier, mobile identifier etc.
- `viewport: str` viewport size.

## Development

```sh
pip install -e '.[playwright]' -r requirements.txt
python -m playwright install chromium
python -m pytest
```
41 changes: 41 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
[build-system]
requires = ["setuptools>=77"]
build-backend = "setuptools.build_meta"

[project]
name = "visual_regression_tracker"
version = "5.0.0"
description = "Open source, self hosted solution for visual testing and managing results of visual testing."
readme = "README.md"
license = "Apache-2.0"
license-files = ["LICENSE"]
requires-python = ">=3.10"
dependencies = [
"requests>=2.32.4",
"dacite>=1.9",
]
classifiers = [
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Topic :: Software Development :: Testing",
]

[project.optional-dependencies]
playwright = ["playwright>=1.50"]

[project.urls]
Homepage = "https://github.com/Visual-Regression-Tracker/Visual-Regression-Tracker"
Source = "https://github.com/Visual-Regression-Tracker/sdk-python"
Issues = "https://github.com/Visual-Regression-Tracker/sdk-python/issues"

[tool.setuptools]
packages = ["visual_regression_tracker"]

[tool.pytest.ini_options]
python_classes = "NoThanks"
junit_family = "xunit2"
asyncio_default_fixture_loop_scope = "function"
15 changes: 5 additions & 10 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,10 +1,5 @@
requests
dacite
playwright >= 1.10
typing-extensions
pytest
pytest-cov
pytest-mock
pytest-asyncio >= 0.24
twine
pylint
pytest >= 9.2
pytest-cov >= 7.1
pytest-mock >= 3.17
pytest-asyncio >= 1.4
pylint >= 4.1
9 changes: 0 additions & 9 deletions setup.cfg

This file was deleted.

23 changes: 0 additions & 23 deletions setup.py

This file was deleted.

Loading
Loading