Skip to content

fix(tests): repoint 3 hourly-review-repair test files at the daily-cadence redesign - #1875

Closed
seonghobae wants to merge 3 commits into
mainfrom
fix/hourly-review-recovery-stale-cron-tests
Closed

fix(tests): repoint 3 hourly-review-repair test files at the daily-cadence redesign#1875
seonghobae wants to merge 3 commits into
mainfrom
fix/hourly-review-recovery-stale-cron-tests

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

Summary

hourly-review-repair.yml was renamed to "Daily Review Recovery" and redesigned from a single hourly cron shared by up to 18 thin per-repository callers into one file with 17 distinct daily cron entries (one <minute> <hour> * * * per repository, staggered across UTC hours 0-16 — one hour intentionally shared by fast-mlsirm and metering-billing-platform). Three test files were never updated to match and were failing on a clean origin/main checkout:

  • tests/test_hourly_review_repair_callers.py_EXPECTED_TARGETS' 17 schedule keys and the schedule-list assertion still used the old hourly "<minute> * * * *" shape instead of the actual daily "<minute> <hour> * * *" cron strings (1 + 16 failing cases).
  • tests/test_github_hourly_conflict_repair.py::test_central_repository_has_hourly_self_caller — asserted cron: "21 * * * *" instead of the central .github self-caller's actual cron: "21 6 * * *".
  • tests/test_pr_review_autofix_nvidia_nim_contract.py::test_review_fix_caller_runs_once_each_hour — asserted a single hourly cron: "23 * * * *" that no longer exists. This one was independently renamed to test_review_fix_caller_keeps_the_github_daily_recovery_slot and repointed at Clearfolio's actual cron: "23 7 * * *" by fix(actions): pin opencode-review-dispatch.yml off starved ubuntu-latest #1870 (merged while this PR was in progress); this branch merges that in and extends the docstring/assertions (reusable-scheduler product-neutrality check) while keeping fix(actions): pin opencode-review-dispatch.yml off starved ubuntu-latest #1870's function name and content as the source of truth.

Found while investigating a live required-check failure on ContextualWisdomLab/.github#1870: agent-review-runtime-quality-ci.yml's "Verify scheduler and contextual-orchestrator review-repair contracts" step runs pytest with no positional test path argument, so it falls back to full default tests/ discovery — picking up all 19 of these stale assertions on every PR. With them fixed, the exact unscoped pytest command the workflow runs passes cleanly, so no change to agent-review-runtime-quality-ci.yml itself was needed.

Also updates:

Test plan

  • PYTHONPATH=. python3 -m pytest tests/test_hourly_review_repair_callers.py tests/test_github_hourly_conflict_repair.py tests/test_pr_review_autofix_nvidia_nim_contract.py -v → all pass (59 tests).
  • Exact unscoped command from agent-review-runtime-quality-ci.yml's coverage step (pytest -q --cov=scripts.ci.pr_review_conflict_scope --cov=scripts.ci.pr_review_autofix_context --cov=scripts.ci.zdr_policy --cov=scripts.ci.contextual_orchestrator_review_policy --cov-branch --cov-fail-under=100, full default tests/ discovery) → 100% branch coverage reached; only 3 failures remain, all pre-existing/unrelated (2 real test/workflow-drift bugs in other files, 1 local gh-CLI-missing sandbox limitation — confirmed present identically on a stash-based A/B against baseline).
  • Full suite: PYTHONPATH=. coverage run -m pytest tests --ignore=tests/test_queue_cancellation_open_pr_revalidation.py --ignore=tests/test_queue_cancellation_revalidation.py -q → went from 23 failed / 2828 passed (captured baseline on clean origin/main) to 3 failed / 2849 passed (all 3 pre-existing and unrelated, as above).
  • coverage report --include='scripts/ci/*' (99%) and interrogate scripts/ci (98.3%) — unchanged from baseline, confirming this is a test-only change.
  • git diff --check — clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KPmJErfkcHer4UVEgrQxUX


Generated by Claude Code

…dence redesign

hourly-review-repair.yml was redesigned from a single hourly cron shared by
18 thin per-repository callers into one file with 17 distinct daily cron
entries (one <minute> <hour> * * * per repository, staggered across UTC
hours 0-16). Three test files were never updated to match and were failing
on a clean origin/main checkout:

- tests/test_hourly_review_repair_callers.py: _EXPECTED_TARGETS' 17 schedule
  keys and the schedule-list assertion still used the old hourly
  "<minute> * * * *" shape instead of the actual daily
  "<minute> <hour> * * *" cron strings (1 + 16 failing cases).
- tests/test_github_hourly_conflict_repair.py::test_central_repository_has_hourly_self_caller
  asserted cron: "21 * * * *" instead of the central .github self-caller's
  actual cron: "21 6 * * *".
- tests/test_pr_review_autofix_nvidia_nim_contract.py::test_review_fix_caller_runs_once_each_hour
  asserted a single hourly cron: "23 * * * *" that no longer exists; renamed
  to test_review_fix_caller_runs_once_each_day and repointed at Clearfolio's
  actual cron: "23 7 * * *", keeping the original protective intent (the
  caller still dispatches to the reusable, product-neutral
  pr-review-fix-scheduler.yml engine).

Found while investigating a live required-check failure on
#1870: agent-review-runtime-quality-ci.yml's
"Verify scheduler and contextual-orchestrator review-repair contracts" step
runs pytest with no positional test path (full default tests/ discovery),
so it was picking up these 19 stale assertions. With them fixed, the exact
unscoped pytest command the workflow runs passes cleanly (100% branch
coverage across pr_review_conflict_scope, pr_review_autofix_context,
zdr_policy, and contextual_orchestrator_review_policy) -- no change to
agent-review-runtime-quality-ci.yml itself was needed.

Full suite (tests --ignore test_queue_cancellation_{open_pr_revalidation,revalidation})
went from 23 failed/2828 passed to 3 failed/2848 passed; the 3 remaining
failures are pre-existing and unrelated (2 real test/workflow-drift bugs in
other files, 1 local gh-CLI-missing sandbox limitation), confirmed identical
before and after on a stash-based A/B. scripts/ci coverage (99%) and
interrogate (98.3%) are unchanged from baseline, confirming this is a
test-only change.

Also updates CHANGELOG.md (first Unreleased section) and
docs/product-technical-gap-baseline.md with a new dated entry recording the
fix; the specific "opencode-review-dispatch.yml ... starved floating image"
entry this was expected to append to was not found verbatim in the current
file, so a new standalone entry was added instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPmJErfkcHer4UVEgrQxUX
…overy-stale-cron-tests

# Conflicts:
#	docs/product-technical-gap-baseline.md
#	tests/test_pr_review_autofix_nvidia_nim_contract.py
@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 3 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: e9d4f92f-ca7f-4f9b-9757-97927d243d88

📥 Commits

Reviewing files that changed from the base of the PR and between b5efbc2 and c1bf559.

📒 Files selected for processing (3)
  • CHANGELOG.md
  • docs/product-technical-gap-baseline.md
  • tests/test_pr_review_autofix_nvidia_nim_contract.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

seonghobae added a commit that referenced this pull request Sep 5, 2026
…st (#1877)

* fix(tests): repair changed-scope drift and stale noema cancel-step test

Two pre-existing tests/ failures blocked the unscoped pytest discovery
run in agent-review-runtime-quality-ci.yml's review-repair contracts
step, unrelated to the hourly-cron fixes in #1875:

- strix.yml's changed-scope job if: had drifted onto a multi-line `>-`
  block scalar when PR #1869 added converted_to_draft handling, and the
  extra continuation lines broke byte-identical parity with
  security-scan.yml/sast-semgrep.yml's copies. Collapsed back to one
  physical if: line with the same expression.
- test_noema_close_cleanup_selects_only_the_closed_pr_across_shared_display_titles
  still targeted the pre-#1869 step name/env vars
  (CLOSED_PR_NUMBER) that PR #1869 renamed to "...for the inactive pull
  request" / INACTIVE_PR_NUMBER/INACTIVE_PR_HEAD_SHA/PR_ACTION when it
  generalized noema-review.yml's cleanup to also cover
  converted_to_draft and added a live_target_matches re-verification.
  tests/test_noema_review_gate.py's equivalent tests were already
  updated; this one was missed. Updated the step name/env vars and
  taught the fake gh to answer the new live-PR lookup -- the PR #1507
  pull_requests[] cancellation-scoping invariant it protects is
  unchanged and still correctly implemented in production.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPmJErfkcHer4UVEgrQxUX

* fix(tests): mock fetch_pr in Strix rerun job selection test

test_dispatch_strix_reruns_scan_job_not_sibling_publisher only mocked
rerun_actions_job, leaving dispatch_strix_evidence's live_dispatch_head_matches
call to invoke the real fetch_pr. In any environment with a real gh CLI on
PATH this hits the actual GitHub API for a synthetic PR that does not exist
there, returning a live/head mismatch ("stale_head") instead of the expected
"rerun"; without gh at all it fails even earlier with a missing executable.

Add monkeypatch.setattr(sched, "fetch_pr", lambda *_args: [pr]) so the
live-head re-read observes the same fixture pr as authoritative, consistent
with how every other GitHub call in this test path is already isolated.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPmJErfkcHer4UVEgrQxUX

* fix(tests): align Strix admission assertion

* fix(tests): align daily review recovery fixtures

---------

Co-authored-by: Claude <noreply@anthropic.com>

Copy link
Copy Markdown
Contributor Author

Redundancy check against current main (b5efbc27, #1877) and mergeable_state: dirty on this PR's own head:

Fully covered by #1877 (verbatim match):

  • tests/test_hourly_review_repair_callers.py: _EXPECTED_TARGETS' all 17 daily-cron keys ("2 0 * * *""49 12 * * *""59 16 * * *") match this PR's proposed values exactly, hour-for-hour.
  • tests/test_github_hourly_conflict_repair.py: the central self-caller assertion is on main as cron: "21 6 * * *" — but under a renamed function, test_central_repository_has_daily_self_caller (not this PR's test_central_repository_has_hourly_self_caller), with more assertions than this PR added.

Already fixed independently before either PR, via #1870 (not #1877):

Gap — not covered anywhere on main:

So: overlap is partial, not complete. The _EXPECTED_TARGETS/central-self-caller content is superseded by #1877; this PR still adds the two extra nvidia-nim-contract assertions and the doc corrections that nothing else has landed. Leaving open per this repo's closure-is-merge-only convention (docs/product-technical-gap-baseline.md / #1397) — recommend rebasing onto current main to drop the now-redundant hunks and keep only the still-missing pieces.


Generated by Claude Code

Resolves the dirty mergeable_state caused by #1877 independently
re-authoring most of this PR's fixes on main.

- tests/test_hourly_review_repair_callers.py and
  tests/test_github_hourly_conflict_repair.py: dropped this branch's
  now-redundant duplicate hunks (same cron assertions fixed to the same
  values) in favor of main's versions verbatim, including main's
  test_central_repository_has_daily_self_caller rename.
- tests/test_pr_review_autofix_nvidia_nim_contract.py: kept the two
  assertions genuinely not covered anywhere on main -- the negative
  cron: "23 */2 * * *" check and the product-neutrality check that
  ContextualWisdomLab/clearfolio is never hard-coded into
  pr-review-fix-scheduler.yml -- layered onto main's already-fixed
  test_review_fix_caller_keeps_the_github_daily_recovery_slot (#1870).
- CHANGELOG.md: kept both this branch's and main's Unreleased bullets
  (additive convention), rewriting this branch's bullet to correctly
  attribute the now-shared-on-main fixes to #1870/#1877 and describe
  only the residual scope this PR still closes.
- docs/product-technical-gap-baseline.md: corrected the
  "opencode-review-dispatch.yml still requesting the starved floating
  image" entry's stale "not yet fixed" sub-note -- the core cron
  assertion was already fixed by #1870 itself, and the other two files
  were fixed on main via #1877. The entry now states this accurately
  and lists the two extra assertions as the actual remaining scope.

Validated: targeted 3-file suite (59 passed), full suite minus the two
excluded queue-cancellation-revalidation files (2833 passed, 1 skipped),
git diff --check clean. Final diff vs origin/main is limited to
CHANGELOG.md, docs/product-technical-gap-baseline.md, and the two new
assertions in tests/test_pr_review_autofix_nvidia_nim_contract.py.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPmJErfkcHer4UVEgrQxUX

Copy link
Copy Markdown
Contributor Author

Merged origin/main and resolved the dirty conflict (now mergeable_state: blocked, i.e. only checks/review pending — no merge conflict).

Resolution:

  • tests/test_hourly_review_repair_callers.py and tests/test_github_hourly_conflict_repair.py: dropped this branch's now-redundant duplicate hunks (both branches fixed the same cron assertions to the same values) and took main's versions verbatim, including main's test_central_repository_has_daily_self_caller rename. Both files are now byte-identical to main.
  • tests/test_pr_review_autofix_nvidia_nim_contract.py: confirmed the core cron assertion (test_review_fix_caller_keeps_the_github_daily_recovery_slot, cron: "23 7 * * *") was already fixed on main by #1870 itself. Kept only the two assertions genuinely not present anywhere on main: the negative cron: "23 */2 * * *" check, and the product-neutrality check that ContextualWisdomLab/clearfolio is never hard-coded into pr-review-fix-scheduler.yml.
  • CHANGELOG.md: kept both bullets (additive convention) and rewrote this branch's bullet to correctly attribute the now-shared fixes to #1870/#1877 and describe only the residual scope.
  • docs/product-technical-gap-baseline.md: corrected the "opencode-review-dispatch.yml still requesting the starved floating image" entry's stale "not yet fixed" sub-note — it now states the core assertion was fixed by #1870 and the other two files by #1877, listing the two extra assertions as this PR's actual remaining scope.

Remaining genuine scope of this PR (diff vs origin/main is now 3 files, +49/-8): the two new regression assertions above, plus the CHANGELOG.md/docs/product-technical-gap-baseline.md corrections.

Validation: targeted 3-file suite (59 passed), full suite minus the two excluded queue-cancellation-revalidation files (2833 passed, 1 skipped, no failures), git diff --check clean. Pushed as c1bf559b.


Generated by Claude Code

@seonghobae

Copy link
Copy Markdown
Contributor Author

Closing as fully redundant with current main. tests/test_pr_review_fix_hourly_contract.py::test_clearfolio_caller_runs_once_each_day already asserts the exact 23 7 * * * daily slot, and test_reusable_scheduler_has_no_product_specific_timer plus tests/test_hourly_review_repair_callers.py::test_no_target_repository_is_hard_coded_in_the_shared_scheduler already enforce the reusable scheduler’s product-neutrality across all consolidated targets. The extra retired-cron negative assertion does not add a distinct production invariant beyond the exact schedule-map contract merged in #1877. Avoiding the 49-line duplicate also prevents another full required-workflow fan-out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants