Skip to content

build: add make reproduce for README evidence - #14

Merged
CoreyLeath-code merged 7 commits into
mainfrom
build/reproducible-research-artifacts
Aug 6, 2026
Merged

build: add make reproduce for README evidence#14
CoreyLeath-code merged 7 commits into
mainfrom
build/reproducible-research-artifacts

Conversation

@CoreyLeath-code

@CoreyLeath-code CoreyLeath-code commented Aug 6, 2026

Copy link
Copy Markdown
Owner

Summary

Adds a single make reproduce entry point that regenerates the benchmark, coverage, quality-check, and repository-inventory evidence cited by the README. The README now points readers to generated artifacts instead of retaining static timing or coverage claims that drift from the commit being reviewed.

Files changed

  • Makefile — adds the reproduce target.
  • scripts/reproduce.py — runs Black, Ruff, mypy, pytest with XML coverage, the 500-iteration benchmark harness, JSON validation, and a tracked-file inventory; writes reproducibility-results.json.
  • .github/workflows/ci.yml — adds an independent job that runs make reproduce and uploads all three artifacts.
  • README.md — replaces static measured values with artifact paths and the exact reproduction command. Design targets remain explicitly labeled as targets, not measurements.

Validation performed

  • CI run 134 passed the existing CI job and the new Reproduce README evidence job.
  • make reproduce passed Black, Ruff, mypy, pytest/coverage, the benchmark harness, and JSON validation.
  • Generated artifact: ride-sharing-reproducibility-artifacts.

README metric reconciliation

Metric README value Regenerated value Match? Artifact
Event-bus benchmark Generated on demand 0.018334 ms average publish; 500 delivered N/A — static claim removed benchmark-results.json
Matching benchmark Generated on demand 0.073836 ms average; driver-10 selected N/A — static claim removed benchmark-results.json
Location/pricing benchmark Generated on demand 0.003317 ms / 0.003672 ms; 1.44x surge N/A — static claim removed benchmark-results.json
Repository inventory Generated on demand 64 tracked files; 32 Python; 5 tests; 3 workflows; 4 manifests Yes — generated from this commit reproducibility-results.json
Line coverage Generated on demand 53.78% Yes — generated from this commit coverage.xml and reproducibility-results.json
Formatting, linting, typing Command outcome All passed Yes reproducibility-results.json

Risks

  • Timing values remain environment-sensitive; they are intentionally published as generated artifacts rather than stable README claims.
  • The target requires Git, Make, and the documented Python runtime/development dependencies.

Follow-up recommendations

  • Review artifacts from the workflow run alongside the exact commit before citing performance or coverage externally.
  • Pin development tooling if bit-for-bit repeatability across environments becomes a project requirement.

@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@CoreyLeath-code, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 38 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: bd32d974-bf88-4f9f-bfa4-f5cc33124414

📥 Commits

Reviewing files that changed from the base of the PR and between 6cb624b and d433fad.

📒 Files selected for processing (3)
  • Makefile
  • README.md
  • scripts/reproduce.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@CoreyLeath-code
CoreyLeath-code marked this pull request as ready for review August 6, 2026 03:43
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@CoreyLeath-code
CoreyLeath-code merged commit 15b3870 into main Aug 6, 2026
12 checks passed
@github-actions

Copy link
Copy Markdown

🎉 This PR is included in version 1.1.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant