Skip to content

[qa][infra] rc3 VM sweep quota-aborted (HTTP 429 session limit) — invalid as a measurement #845

@100yenadmin

Description

@100yenadmin

Not a product result — recording for honesty + traceability.

The rc3 VM sweep (@a245a2c) hit the claude account session limit (HTTP 429) mid-run:

No scores.db row written (would be junk). Results archived on the VM at results.rc3-QUOTA-ABORTED-*.

Root cause (operational): a clean 5-persona Opus release sweep needs ~5 cold-opens + ~200 beats of Opus generation in one account session-quota window — and that window is shared with the AI's own heavy agent/workflow usage. The 4× parallel cold-opens burst-burned the remainder and tripped the limit.

Fix landed: PR #844 — the sweep now (a) runs personas sequentially so the first 429 aborts before the rest spend, (b) detects the 429 and emits an explicit ABORTED status instead of a number, (c) attributes infra-abort vs product-failure in the rollup. So a future quota-storm can never masquerade as an RRI again.

Re-run: the hardened sequential sweep at current main (not a245a2c — main has moved to #834+) after the quota resets / on a quota-isolated window. Tracks toward the first clean non-partial v1.0.4 RRI.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions