PyBencher

PyBencher is a simple, decorator-based benchmarking suite for Python. It provides detailed timing statistics (average, median, standard deviation) and supports per-test configuration overrides.

Installation

pip install pybencher

Basic Usage

from pybencher import Suite

suite = Suite()

# Quick registration
@suite.bench()
def my_function():
    return sum(range(10000))

# Per-benchmark configuration
@suite.bench(name="Fast Math", max_samples=5000)
def fast():
    return 1 + 1

# Function arguments via args / kwargs
@suite.bench(args=(10, 20), name="Add")
def add(a, b):
    return a + b

# Manual registration (equivalent to @suite.bench)
def manual_func(n):
    return sum(range(n))

suite.add(manual_func, args=(1000,), name="Manual Register")

# Run and print results
results = suite.run()
results.print()

Configuration Overrides

Any setting in the Suite can be overridden per-benchmark via keyword arguments to bench() or add(). Function inputs are separated cleanly into the args and kwargs parameters.

Override	Type	Description
name	str	Display name in reports
timeout	float	Per-test time limit in seconds
max_samples	int	Maximum statistical samples to collect
min_samples	int	Minimum statistical samples to collect
cut	float	Percentage of outliers to trim (0.0 to 0.5)
disable_stdout	bool	Mute print() output inside the target
disable_gc	bool	Disable garbage collection during benchmark
batch_size	int	Force a specific batch size (0 for auto-calibration)
live_output	bool	Show real-time progress in the console
verbose	bool	Include extra stats in results.print()
before	callable	Local setup hook
after	callable	Local teardown hook
args	tuple	Positional arguments for the target function
kwargs	dict	Keyword arguments for the target function

Reference

Suite

• timeout (float): Default time limit (10s).
• max_samples (int): Default max samples (1000).
• min_samples (int): Default min samples (3).
• cut (float): Default outlier threshold (0.05).
• warmup_samples (int): Number of unmeasured runs to perform before benchmarking (0).
• validate_responses (bool): Enable cross-test output consistency checks (False).
• validate_limit (int): Max number of iterations to store for full sequence validation (5).
• disable_stdout (bool): Global stdout suppressor.
• disable_gc (bool): Global GC control.
• batch_size (int): Global batch size override.
• live_output (bool): Global progress toggle.
• verbose (bool): Global verbosity flag.

BenchmarkResults

• print(verbose=None): Print results to console.
• to_json(indent=4): Export results to JSON.
• to_markdown(): Export results as a GitHub-flavored markdown table.
• to_list(): Export results to list of dicts.

BenchmarkResult

Dataclass containing:

• name: Target name or custom override.
• avg, std, median, minimum, maximum: Timing stats in seconds.
• itr_ps: Iterations per second.
• iterations: Total runs.
• counted_iterations: Runs used for stats after trimming outliers.

Example

from pybencher import Suite

suite = Suite()

# 'timeout' here is a benchmark config override, not a function param
@suite.bench(args=(0.1,), name="Sleepy", verbose=True)
def test_sleep(duration):
    import time
    time.sleep(duration)

# Function kwargs stay separate from benchmark config
@suite.bench(kwargs={"timeout": 0.1}, name="Sleepy Alt", verbose=True)
def test_args(timeout):
    import time
    time.sleep(timeout)

results = suite.run()
results.print()

Output:

Sleepy: 100ms/itr | 10.0 itr/s
  std:     120us
  median:  100ms
  min/max: 99ms / 101ms
  runs:    10 (10 counted)
  total:   1.01s

CI/CD Pipeline

PyBencher uses an automated GitHub Actions pipeline:

• Testing: Every push to any branch triggers a full test suite across Linux, Windows, and macOS for Python 3.10–3.14.
• Publishing: Pushes to main automatically publish to PyPI if and only if all tests pass.