Skip to content

Fix strict cost accounting for verified Copilot context parser refusals - #114

Open
fanyangCS wants to merge 6 commits into
mainfrom
fix/113-copilot-context-parser-accounting-20260908
Open

Fix strict cost accounting for verified Copilot context parser refusals#114
fanyangCS wants to merge 6 commits into
mainfrom
fix/113-copilot-context-parser-accounting-20260908

Conversation

@fanyangCS

@fanyangCS fanyangCS commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes #113.

A top-level Copilot CLI parser failure (error: unknown option '--context') could leave an error usage record with unknown cost. Under strict cost control, that unresolved record then blocked subsequent calls across projects even after the CLI mismatch was repaired. Existing pre-provider refusal handling did not recognize this diagnostic, and the missing session ID prevented normal exact-session usage reconciliation.

Fix

  • Recognize only the exact runner-wrapped --context parser diagnostic, corroborated by a matching host-generated agent.io.complete receipt: Copilot run_exec, correlated call and run label, exit 1, failed/uncompleted turn, no session, no assistant/stdout/JSON/tool activity, and --context default in the command arguments.
  • Pass the host completion receipt through new-call finalization. For affected historical records, lazily retrieve the call-correlated completion from the project event log and reuse the existing read-time not_billed / not_started / zero-cost projection. Missing or duplicate completion receipts fail closed; original ledger bytes are not rewritten and execution status remains error.
  • Preserve metering guards: token fields (including explicit zero), premium usage, nano-AIU, model usage, and explicit cost cannot be waived by this new classification. Arbitrary quoted parser text, post-start activity, network failures, and genuinely unknown costs remain unresolved.

No broad unknown option substring rule, budget-policy relaxation, ledger deletion, or new reconciliation subsystem is introduced.

The source digest change also requires regenerated release artifacts. A separate follow-up commit runs the supported python -m argus_skill.release_tools.build_release pipeline to refresh the JSON release manifest, generated TypeScript release identity, and shipped web/TUI bundles. No handwritten frontend behavior or dependency lockfiles are changed.

Verification

  • Re-ran the four targeted suites after fast-forwarding the feature branch onto current main:
    python -m pytest tests/test_agent_cli_backend.py tests/core/test_copilot_parser_refusal.py tests/core/test_usage.py tests/core/test_cost_control.py
    164 passed. Coverage includes the actual adapter finalization boundary, historical idempotence and unchanged ledger bytes, cross-project admission, missing/duplicate/foreign receipts, quoted/post-start errors, observed usage, and a remaining genuine unknown-cost blocker.
  • ruff check argus_skill tests and git diff --check passed.
  • Supported release build, generate_manifest --check, and check_artifacts passed; the same 164 targeted tests passed again after the artifact refresh. This corrects the first CI run's stale-manifest failure.
  • Before publication, the local fix was also exercised through a normal bounded CLI mission using the unchanged global accounting root: Manager grounding → Engineer → independent Reviewer → Manager stage certification completed successfully. The original global unresolved count changed from 1 to 0 without rewriting the original ledger or relaxing the strict policy; final active reservations were 0. All four real model calls retained normal priced usage accounting. These are Argus accounting metrics, not evidence of an additional cash charge.

Limits / separate work

  • The real smoke mission succeeded on initial grounding; it did not force the grounding-retry branch. The parser-failure/retry classification is covered by regression fixtures and the historical failure receipt.
  • Copilot initial call and retry use different CLI versions, causing retry failure #112 is the separate CLI selection/version mismatch problem. This PR reuses the already-repaired local environment and does not fix routing, capability discovery, or version selection.
  • This intentionally handles the evidenced --context default parser failure, not every possible CLI startup error. It does not change the user-facing budget_exhausted wording.
  • No long-running services were restarted, so adoption by an already-running service is not claimed. Full cross-platform CI results are reported by the checks on this draft PR.

@fanyangCS
fanyangCS requested a review from lbx154 September 8, 2026 03:59
@fanyangCS
fanyangCS marked this pull request as ready for review September 8, 2026 04:12
@fanyangCS
fanyangCS requested a lite review from Copilot September 8, 2026 04:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The change is narrowly scoped, fails closed without a matching trusted completion receipt, and is covered by focused regression tests for both new and historical records.

Pull request overview

This PR fixes strict cost-control deadlocks caused by a specific Copilot CLI top-level argument parser failure (unknown option '--context') by classifying only verified, pre-turn, silent failures as unbilled using a trusted agent.io.complete host receipt (including for historical ledger reads), while preserving all existing metering guards.

Changes:

  • Add narrow recognition of the --context default parser diagnostic and corroborate it via agent.io.complete receipt checks to safely project not_billed/not_started/$0 when no usage was observed.
  • Thread the host completion receipt through exec finalization and lazily load call-correlated receipts from events.jsonl when re-reading historical usage records.
  • Add regression tests for new-call finalization, historical idempotence, and fail-closed behavior; regenerate release artifacts / web bundles tied to the source digest.
File summaries
File Description
tests/test_agent_cli_backend.py Adds an integration-style adapter test verifying receipt-backed classification behavior with/without observed output.
tests/core/test_copilot_parser_refusal.py New targeted regression suite for Issue #113, including historical projection/idempotence and fail-closed cases.
frontend/web/dist/index.html Regenerated web entrypoint asset hash reference.
frontend/web/dist/assets/square-Vz4OpK6j.js Regenerated bundle chunk to import the updated hashed index module.
frontend/web/dist/assets/MapPanel-B55b--j_.js Regenerated bundle chunk updated to reference the refreshed hashed dependencies.
frontend/core/src/release.generated.ts Regenerated release ID/source digest constants.
argus_skill/release_manifest.json Regenerated release manifest with updated release ID/source digest.
argus_skill/core/usage.py Implements receipt-assisted historical projection and new-call startup receipt plumbing for parser refusal classification.
argus_skill/core/runner_errors.py Adds exact-match parser diagnostic detection + strict receipt corroboration helper.
argus_skill/adapters/agent_cli_backend/_exec_spawn.py Logs agent.io.complete summary and passes it through as startup_receipt.
argus_skill/adapters/agent_cli_backend/_exec_finalize.py Accepts/persists startup_receipt through usage record construction.
Review details
  • Files reviewed: 8/15 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

lbx154 added a commit that referenced this pull request Sep 8, 2026
Keep grounding corrections on the same ACP client; align one-shot loader policy with readiness and remove redundant context defaults. Preserve post-start ACP failures and usage without replaying them.

Reuse and harden the reviewed PR #114 implementation: correlate only exact parser refusals with silent host completion receipts, preserve observed metering in both ledger and receipt, and project historical non-billed startup failures without rewriting source records.

Co-authored-by: fan yang <5820832+fanyangCS@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Copilot CLI startup failure leaves unknown cost and blocks subsequent calls

2 participants