Skip to content

gh-995: Allow filtering a single benchmark test#1068

Merged
connoraird merged 2 commits into
mainfrom
connor/issue-995
May 12, 2026
Merged

gh-995: Allow filtering a single benchmark test#1068
connoraird merged 2 commits into
mainfrom
connor/issue-995

Conversation

@connoraird
Copy link
Copy Markdown
Contributor

@connoraird connoraird commented May 12, 2026

Description

  • Allows the passing of pytest flags to the benchmark and regression tests nox commands

Fixes: #995

Changelog entry

Added: Ability to run benchmarks/regression tests for a single test

Checks

  • Is your code passing linting?
  • Is your code passing tests?
  • Have you added additional tests (if required)?
  • Have you modified/extended the documentation (if required)?
  • Have you added a one-liner changelog entry above (if required)?

@connoraird connoraird self-assigned this May 12, 2026
@connoraird connoraird requested a review from paddyroddy May 12, 2026 10:12
@paddyroddy paddyroddy added enhancement New feature or request benchmarks Benchmarking work labels May 12, 2026
Copy link
Copy Markdown
Member

@paddyroddy paddyroddy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These need to be documented somewhere.

Also do you think it's clearest having one function rather than benchmark and benchmark_all? I know it's not very DRY, but I'm just thinking of other users that aren't us

Comment thread noxfile.py
session.run(
"pytest",
BENCH_TESTS_LOC,
*SHARED_BENCHMARK_FLAGS,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why weren't we using these before?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we just missed this.

@connoraird
Copy link
Copy Markdown
Contributor Author

Also do you think it's clearest having one function rather than benchmark and benchmark_all? I know it's not very DRY, but I'm just thinking of other users that aren't us

I think it's clearer to have benchmarks and pass through the pytest args as that is what we do eslewhere, for example, in `tests.

@connoraird connoraird requested a review from paddyroddy May 12, 2026 13:06
@paddyroddy
Copy link
Copy Markdown
Member

I think it's clearer to have benchmarks and pass through the pytest args as that is what we do eslewhere, for example, in `tests.

Okay, fair enough

Copy link
Copy Markdown
Member

@paddyroddy paddyroddy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@connoraird connoraird merged commit 7e620f7 into main May 12, 2026
22 checks passed
@connoraird connoraird deleted the connor/issue-995 branch May 12, 2026 13:28
@connoraird
Copy link
Copy Markdown
Contributor Author

connoraird commented May 12, 2026

I missed out a call to _setup_array_backend(session) in the noxfile function benchmarks. I'll add it as part of #1064

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

benchmarks Benchmarking work enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add the ability to run regression tests only on certain tests

2 participants