Skip to content

fix(budget): enforce token grants and record provider cost in the ledger - #199

Merged
Glen-SP merged 11 commits into
mainfrom
validation/budget-and-cost-accounting
Sep 15, 2026
Merged

Glen-SP merged 11 commits into
mainfrom
validation/budget-and-cost-accounting

Conversation

@Glen-SP

@Glen-SP Glen-SP commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

Budget enforcement was advisory: a run could overrun its token grant with nothing
throttling it, and no neuron or USD figure ever reached a run artifact. This makes
grant admission actually gate execution, adds BudgetLedger.get_exhausted_units()
as the exhaustion probe, accumulates provider cost metering in the ledger, and
escalates a ResourceBudgetError to severity error even when
require_budget_grant is False.

Informational units (cost_usd, neurons) are tracked for reporting and never
gate execution.

Fixes #81
Fixes #85

Scope

  • Affected modules or public contracts: praxist/core/{budget,execution_guards, ledgers,modeling,protocol,replay,runtimes}.py,
    codex_sdk/_events.py, research_loop/stage.py. Adds one public method
    (get_exhausted_units) and one constant (INFORMATIONAL_USAGE_UNITS).
  • Compatibility considerations: runs that previously overran a grant and completed
    will now be halted at the grant boundary, which is the intended correction rather
    than a regression. Existing ledgers are read unchanged; no migration.
  • Task-agnostic rationale: budget admission and cost accounting are enforced by the
    shared guard layer, independent of any task's metric or protocol.

Verification

Capability validation of the budget gate, ledger and cost accounting, using the
branch's own _write_budget_grant helper so grants are constructed exactly as its
tests construct them:

Probe Setup Result
A granted tokens:100, used tokens:100 exhausted_units == ['tokens'] — guard fires
B granted tokens:100, used tokens:50, cost_usd:99 recorded, exhausted_units == [] — informational tracked, not enforced
C granted tokens:100, cost_usd:1.0, used cost_usd:1.0 initially ['cost_usd'] — inconsistent, fixed in this branch

Probe C found that two modules disagreed about whether informational units are
enforceable: append_usage skipped them when absent from granted_budget, but
this branch also added them to ALLOWED_BUDGET_UNITS, so a grant could legally
contain cost_usd while get_exhausted_units iterated granted_budget with no
informational exclusion — turning a reporting-only unit into a hard run halt. The
exclusion is now explicit in get_exhausted_units, with a regression test.

Full checks, run on this branch merged with current main (a clean merge — no
conflicts; main's recent commits touch none of the files above):

  • ruff check praxist/ tests/ — All checks passed
  • ruff format --check praxist/ tests/ — 450 files already formatted
  • pyrefly check — 0 errors
  • pytest tests/unit --tb=short -q — 2913 passed
  • run_test_coverage.py unit --fail-under 90 --fail-under-statements 95 — passed,
    93.26% total / 95.01% statements
  • run_test_coverage.py integration — passed
  • pytest (full) — 3353 passed, 7 skipped, 285 subtests

Checklist

  • The change is focused and does not include unrelated generated files.
  • Tests cover the affected behavior.
  • Documentation, templates, examples, and skills were updated where needed.
  • No credentials, private task data, or research-run artifacts are included.
  • New dependencies and copied assets include their source and license terms.
  • I have read .github/CONTRIBUTING.md and the contribution terms in LICENSE.md.

MASTER870-CMD and others added 10 commits September 3, 2026 17:59
INFORMATIONAL_USAGE_UNITS (cost_usd, neurons) are tracked for reporting and
must never gate execution. They belong to ALLOWED_BUDGET_UNITS so that
_validate_budget_amounts accepts them as recorded usage, but that also makes
them grantable -- and get_exhausted_units iterated granted_budget without
excluding them. A grant carrying cost_usd would therefore raise
ResourceBudgetError and halt the run on a unit documented as advisory.

Skip informational units in the exhaustion probe so enforcement matches the
documented contract. Enforced units (tokens, wall_clock_seconds, gpu_hours)
are unaffected; the added test asserts a maxed-out cost_usd yields no
exhaustion while tokens still does.
Brings the branch current with main so it satisfies strict required status
checks (branch protection: required_status_checks.strict, enforce_admins).
The two incoming commits are documentation only (#193, #194) and touch no
code, so the branch's Stage 2 result is unaffected.
@Glen-SP
Glen-SP merged commit 9d6932d into main Sep 15, 2026
7 checks passed
@Glen-SP
Glen-SP deleted the validation/budget-and-cost-accounting branch September 15, 2026 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants