Skip to content

Add CodSpeed performance benchmarks for the backend - #1773

Draft
codspeed[bot] wants to merge 1 commit into
developfrom
codspeed/setup-benchmarks
Draft

codspeed[bot] wants to merge 1 commit into
developfrom
codspeed/setup-benchmarks

Conversation

@codspeed

@codspeed codspeed Bot commented Sep 24, 2026

Copy link
Copy Markdown

Summary

This PR sets up continuous performance testing with CodSpeed for the FastAPI backend, using pytest-codspeed in CPU simulation mode.

Changes

  • backend/tests/benchmarks/test_bench_core.py: 10 benchmarks covering CPU-bound backend hot paths:
    • parse_eml_bytes on a realistic multipart email (plain + HTML + CSV attachment, long References header) and on the threading reply fixture
    • strip_html_markup on an HTML document with scripts, entities, comments and angle-bracket emails, plus the plain-text fast path
    • contains_html_markup
    • extract_reference_ids on a long folded References header with duplicates
    • Strong dedupe fingerprints for a batch of 200 import candidates
    • Hybrid retrieval query normalization + TM2C2 score fusion (rankweave) across 500 candidates
    • evaluate_calendar_conflicts_from_ics against a 100-event existing calendar
    • evaluate_access RBAC/ABAC decisions for 300 requests
  • The benchmark module calls pytest.importorskip("pytest_codspeed"), so the existing Application CI run (which installs from requirements-hashes.txt without the plugin) simply skips it and is unaffected.
  • backend/pyproject.toml / backend/uv.lock: pytest-codspeed==5.0.3 added to the dev dependency group (pinned like the other dev dependencies). The runtime hashed requirement files are untouched.
  • .github/workflows/codspeed.yml: new workflow running on PRs and pushes to develop/master (plus workflow_dispatch for backtesting). It follows the existing CI conventions (harden-runner, commit-SHA pinned actions, Python 3.14, persist-credentials: false), installs dependencies with uv sync --frozen --group dev, and runs CodSpeedHQ/action v5.2.1 in simulation mode with OIDC authentication (no token secret needed).
  • README badge and a short "Performance benchmarks" section in backend/README.md.

Running locally

cd backend
uv sync --group dev
uv run pytest tests/benchmarks --codspeed

Next steps

  • Once merged, CodSpeed will build a baseline on develop, and subsequent PRs will get performance reports comparing against it.
  • Add more benchmarks as new CPU-heavy paths appear (e.g. content graph parsing, attachment parsing) by adding @pytest.mark.benchmark tests under backend/tests/benchmarks/.
  • Optionally enable performance checks in the CodSpeed dashboard to block regressions.

@coderabbitai

coderabbitai Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: ContextualWisdomLab/naruon/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 71936b46-5d6b-4fe6-aa16-b69d7c63c005

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codspeed
codspeed Bot marked this pull request as ready for review September 24, 2026 17:50
@codspeed
codspeed Bot requested a review from seonghobae as a code owner September 24, 2026 17:50
@codspeed

codspeed Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Author

Congrats! CodSpeed is installed 🎉

🆕 10 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks


Open in CodSpeed

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode reviewed the current-head product diff. Coverage is a separate gate.

Changed files

  • .github/workflows/codspeed.yml — GitHub Actions review job
  • README.md — repository behavior
  • backend/README.md — API and service runtime
  • backend/pyproject.toml — API and service runtime
  • backend/tests/benchmarks/__init__.py — regression suite
  • backend/tests/benchmarks/test_bench_core.py — regression suite
  • backend/uv.lock — API and service runtime

Changed behavior

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: codspeed.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: codspeed.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Repository file: README.md"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Repository file: README.md"]
  R2 --> V2["required checks"]
  Evidence --> S3["Backend: README.md (3 files)"]
  S3 --> I3["API and service runtime"]
  I3 --> R3["Review risk: Backend: README.md (3 files)"]
  R3 --> V3["backend tests"]
  Evidence --> S4["Test: __init__.py (2 files)"]
  S4 --> I4["regression suite"]
  I4 --> R4["Review risk: Test: __init__.py (2 files)"]
  R4 --> V4["targeted test run"]
Loading

Findings

No source-backed product finding is synthesized from the coverage gate. A coverage miss belongs in the status comment.

  • Head SHA: dad2902ce3ad4fe6657deb98ab3ad4f634778ede
  • Workflow run: 36093653489
  • Workflow attempt: 1
  • Coverage gate: failure

Review outcome

Coverage is a gate, not the review. This body reviews the changed product files.

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: codspeed.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: codspeed.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Repository file: README.md"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Repository file: README.md"]
  R2 --> V2["required checks"]
  Evidence --> S3["Backend: README.md (3 files)"]
  S3 --> I3["API and service runtime"]
  I3 --> R3["Review risk: Backend: README.md (3 files)"]
  R3 --> V3["backend tests"]
  Evidence --> S4["Test: __init__.py (2 files)"]
  S4 --> I4["regression suite"]
  I4 --> R4["Review risk: Test: __init__.py (2 files)"]
  R4 --> V4["targeted test run"]
Loading

@opencode-agent

opencode-agent Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

OpenCode Review Overview

Coverage evidence did not pass, so approval is blocked. The formal pull-request review is the source-backed diff review, not this status comment.

Copy link
Copy Markdown
Contributor

Admission correction — exact current head dad2902ce3ad4fe6657deb98ab3ad4f634778ede was re-fetched immediately before this transition. The PR is Open and its branch/delta are preserved, but it is not merge-admissible: CHANGES_REQUESTED 1; terminal workflow: Security Scan:failure, CodeQL PR:failure. Moving to Draft/Proposed prevents stale Ready state from consuming review/merge admission. No review, commit, thread, or valid delta is discarded; return to Ready only after the same unchanged head has resolved substantive findings, terminal required Checks, and qualifying independent approval.

@seonghobae
seonghobae marked this pull request as draft September 26, 2026 15:19

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants