[Feature]: pytest integration (run tests with performance regression detection)

[KaykCaputo]

Summary

Add pytest integration to OracleTrace to allow running test suites and detecting performance regressions directly from the CLI.

Problem

Currently, OracleTrace works well for standalone scripts, but most real-world Python projects use pytest as their test runner. There is no straightforward way to run tests, compare performance between runs, and enforce regression thresholds within CI using OracleTrace.

Proposed Solution

Introduce support for running pytest via the OracleTrace CLI:

oracletrace run -- pytest

This should:

• Execute pytest as the target workload
• Capture execution traces
• Support comparison between runs
• Fail execution when regressions exceed a threshold
• Be fully compatible with existing flags (--json, --compare, --fail-on-regression, --threshold, --ignore, --top)

Use Case

Typical CI workflow:

• Generate a baseline from a stable branch
• Run tests in a PR
• Compare results
• Fail CI if performance regresses

This enables OracleTrace to act as a performance guardrail in test pipelines.

Example (optional)

# baseline
oracletrace run -- pytest --json baseline.json

# comparison
oracletrace run -- pytest --json current.json --compare baseline.json --fail-on-regression --threshold 25

CI failed: performance regression detected

test_api.py::test_response_time
+38% slower (120ms → 166ms)

Hotspot:
parse_payload() → +52%

Alternatives Considered

• Running OracleTrace only on standalone scripts (does not integrate with real workflows)
• Using pytest-benchmark (requires test-level instrumentation and does not provide execution trace diff)

Additional Context

This feature aligns with the goal of positioning OracleTrace as a "CI guardrail" for performance regressions.

Checklist

• I searched existing issues before opening this request
• I described the problem and why this feature is useful
• I provided enough detail for implementation discussion

[waldo121]

I believe something like this would make more sense since the pytest command could have arguments too

# baseline
oracletrace run --json baseline.json -- pytest [pytest arguments] 

[KaykCaputo]

Nice, i think your approach really makes more sense