Skip to content

feat(evidence): restore bounded request outcome export with retained lineage - #1138

Merged
seonghobae merged 254 commits into
mainfrom
codex/request-outcome-export-integrated-20260912
Sep 17, 2026
Merged

seonghobae merged 254 commits into
mainfrom
codex/request-outcome-export-integrated-20260912

Conversation

@seonghobae

@seonghobae seonghobae commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Operator outcome

Provide a service-admin, prompt-free export joining retained admissions to
durable workflow and batch associations. Missing, invalid and truncated
evidence stays explicit; this does not infer customer correctness.

Lineage and boundary

Stacked on #1107. Preserves its current-main integration and the valid constant
and test from #1126. No predecessor is closed. Archived exporter history is
documented rather than silently replaced; protected-main adoption is unproven.

The existing journal owns atomic workflow linkage and original provenance.
The existing admin authorization path durably audits before reading evidence.
Pagination and per-admission links are bounded. Malformed retained JSON no
longer prevents index migration; invalid records remain stored and counted,
while valid workflow origins remain unique. No numerical estimator, new service,
production routing default or external owner-source dependency is introduced.

Historical verification (prior revisions)

  • Source revision 40d344dc4733e6fa2b8d45817faba8d4ba4f6dbd includes the
    canonical LaRT inventory repair; full suite 3,724 passed, 2 skipped in
    156.29s
    , terminal session 73240. Documentation-only handoff is 342cbbe6. The preceding
    aae29136 run had one inventory failure, 3,723 passed and 2 skipped.
  • Independent installed-core plus native acceptance: 74 passed in 8.47s,
    outside checkout with Python isolated mode. Both import origins asserted
    under the isolated installation, without source namespace injection.
  • All 46 Python package members match the aae29136 archive, built wheel and
    installed bytes. The diff from that revision to 40d344dc contains only
    the runbook and paper inventory; installed acceptance is scoped to unchanged runtime.
  • Core wheel SHA-256:
    419d6bcc7fe5d77493b917d77bbd7207612253179901886bb7bd9e1ff883f5f7.
  • Reused unchanged native wheel SHA-256:
    8dfee5d228a28733136e25c6006f77006bcba095863a667e0f2a3e71ca8c7c04.
  • Root inspected three browser screenshots at 1265 × 712, English: runbook
    top, complete sequence diagram, ADR top. Labels and identifiers were legible
    without overlap/clipping in those views; ADR navigation worked. Lower ADR,
    current mobile and other locales are not accepted by this limited inspection.

Remaining gates

Required current-head reviews/checks, protected-stack delivery and release are
not established. No observed accuracy or decision-latency improvement is
claimed. Real provider integration and independent adjudication are separate
from controlled-output HTTP tests. Returned to Draft at unchanged head 1881ef06 after the strict full-suite audit exposed unresolved repository-wide resource ownership; those gates remain unsatisfied.

See docs/doctoring/request_outcome_export.md and Proposed ADR 0131 for the
RED/fix history, assumptions, limits, transaction behavior and rollback.

Current repair evidence — 1881ef0

Normal-pushed head 1881ef06ed90ee72eb7209434db366c851cd68dc contains source/test repair 7ebf577535139c6655a1b8d360682511defc22ba and evidence documentation. It normally merges #1107 head de21ffd42733d6115800b33c4e84b3626a33f758 through d1a080d7; no predecessor was closed and no force push was used.

  • Same-head strict full-suite audit: 1,287 failed, 2,472 passed, 2 skipped, 15 errors in 289.06s with .venv/bin/python -m pytest -q -W error --tb=short; log /tmp/co-export-full-strict-1881ef06.log. The process ended, but the original session receipt was lost during context compaction, so a separately observed exit code is not claimed. These are test outcomes, not 1,287 independent root causes. Tracebacks include unclosed HTTPError responses, listener sockets and SQLite connections surfaced as PytestUnraisableExceptionWarning. Repository-wide resource ownership remains unresolved. Neither focused 124-test strict passes nor the default-warning full pass below establish strict full-suite success. No blanket warning filter or global finalizer was added.
  • Same-head whole-suite validation at 1881ef06ed90ee72eb7209434db366c851cd68dc: 3,762 passed, 2 skipped in 118.22s, terminal session 47002, exit 0; log /tmp/co-export-full-suite-1881ef06.log. Command: .venv/bin/python -m pytest -q --tb=short, using the workflow-equivalent default warning policy (not -W error). This is distinct from the focused/native and fresh-installed 124-test warnings-as-errors results below.
  • Same-head Rustdoc coordinate counterexample: 1 passed in 3.05s with rustdoc --test docs/doctoring/irt_router_measurement_review.md. This is a synthetic unit-coordinate counterexample, not empirical parameter fitting or customer accuracy/latency KPI gain.
  • Disabled measurements now return 503 after authorization and before any export query (addresses discussion_r3995903985).
  • Same-cutoff initial/final receipt projection preserves canonical provenance and null missing durations. The 257-admission regression verifies unique 200+57 pagination and excludes late acknowledgement.
  • Independent review found two malformed-phase gaps; nine RED regressions now pass. Initial selected/u64/no-ack validation and explicit invalid-time counting prevent malformed measurement evidence.
  • Actual-native focused group: 62 passed, exit 0, warnings-as-errors. Six-module expansion: 124 passed in 4.85s, exit 0.
  • 23 inherited failure-union nodes were first compared in 46 fresh parent/candidate processes. After authorized test-only SQLite/listener root cleanup, all 23 pass independently. No warning suppression or forced GC. Direct-script test execution is not covered by the pytest cleanup fixture.
  • Fresh offline core/native wheel builds passed the disjoint manifest check. A new noneditable environment outside checkout asserted both site-packages import origins and passed the same 124 tests in 50.30s using isolated Python and warnings-as-errors.
  • Core SHA-256: 456b8e53b075cd6c4654eda23c580999a911bc1f3ff7f0e98fcbe85c73f769bd.
  • Native SHA-256: a2d9456d11ed7032b76a15c2e5562b202c6aad9521285b312073c9dff5b6d55b.
  • Direct browser screenshot inspection covered the changed runbook, AGENTS/CLAUDE/Gap sections, and complete repair source diff at requested 1265 × 712, English/default. This is not full product UI/locale acceptance.

See current validation runbook. Historical full-suite evidence above does not certify this new head. Current-head hosted checks, independent approval, protected merge, release/deployment, real provider adjudication, and customer accuracy/decision-latency gains remain unproven.

seonghobae and others added 30 commits September 8, 2026 10:41
Buyer accuracy claims still require observed task outcomes. Latency claims
require request and decision traces, not outcome labels.
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
…n target

Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>

Copy link
Copy Markdown
Contributor Author

2026-09-12 stack refresh: exact head 8de04219e2ce58c6f88504d8cb6c7f41056293cd is now non-mergeable against stacked base autoresearch/20260909-kpi-loop@9863638dd0915737d45a4265fb029bf7453add5c. Preserve the request-outcome export delta and keep the dependent open; repair the unresolved admission-measurement/unavailable P1 first, then non-force merge-forward/restack onto the canonical parent and obtain fresh exact-head Checks. A conflict or Draft state is not successor completion.

…-export-integrated-20260912

# Conflicts:
#	docs/product-technical-gap-baseline.md
@seonghobae

Copy link
Copy Markdown
Contributor Author

Restack onto main

New SHA: 6075bb5f (merged origin/main @ 767e67fb into codex/request-outcome-export-integrated-20260912 @ 1881ef06, non-force push).

Conflicts: one, in docs/product-technical-gap-baseline.md (dated entries) — kept both, ours first, no code conflicts. contextual_orchestrator/orchestrator.py, server.py, nim_benchmark.py, api_contract.py, pyproject.toml, uv.lock all auto-merged cleanly.

Tests

Note: this repo ships a Rust-native extension (contextual_orchestrator/_decision_receipt, built via maturin from rust/decision_receipt/) that decision_receipts.py imports unconditionally. It was not pre-built in this worktree; I built it locally (maturin develop --release) before running tests — otherwise every decision-receipt test fails with ModuleNotFoundError. Flagging this in case other CI/dev environments hit the same first-run gap.

  • PR's own touched tests (test_batch_request_lineage.py, test_cost_request_cleanup.py, test_cost_review_server.py, test_decision_cache_aggregation.py, test_decision_receipts.py, test_decision_wheel_manifest.py, test_nim_benchmark_release_acceptance.py, test_paginated_decision_provenance.py, test_paper_contracts.py, test_persistence.py, test_repository_security_metadata.py, test_request_outcome_associations_1114.py, test_request_outcome_export.py, test_workflow_request_link.py): 245 passed, 0 failed (after building the native extension above).
  • tests/test_api_contract.py tests/test_self_check.py: 9 passed.
  • python -m interrogate -v contextual_orchestrator/: 100.0% (PASSED).

Readiness review

The delta matches the title: a bounded, service-admin, prompt-free export joining retained admissions to durable workflow/batch associations, with explicit handling of missing/invalid/truncated evidence rather than inferred correctness. decision_receipts.py (new) binds the Rust DecisionReceipt clock to the HTTP request lifecycle; each write path (accepted_request, selection_attempt, initial_decision, provider_dispatch) is wrapped so a store failure is logged (type only, no exception content — matches the description's "no secret leakage" claim) and never silently upgrades to a false ack. No obvious defects found in the reviewed paths (decision_receipts.py, the export/pagination code exercised by test_request_outcome_export.py and test_paginated_decision_provenance.py).

The PR body itself is candid about outstanding gates and is worth restating: it was explicitly returned to Draft at the (then) current head after a strict full-suite (-W error) audit found repository-wide resource-ownership warnings (unclosed HTTPError/listener/SQLite handles) — described there as "unresolved" and separate from this PR's own scope. That strict-suite state is unchanged by this restack (I did not re-run the full strict suite, only the PR's targeted tests and the required contract/self-check gates, both green). Required current-head hosted reviews/checks and protected-stack delivery are also explicitly not yet established per the description.

Verdict: NEEDS_WORK (per the PR's own admission, not a new finding from this review) — concrete outstanding items before it's a real merge candidate:

  1. Repository-wide resource-ownership cleanup (unclosed HTTPError/listener/SQLite handles) surfaced by the strict full-suite run, called out in the description as the reason it was returned to Draft.
  2. Full strict-suite (-W error) result at the new restack head has not been (re-)established — only targeted/contract tests were run here.
  3. Hosted CI checks and independent review/approval at the new head are outstanding.

No new defects found in the diff itself; the merge was clean apart from the doc file, and every test I ran passed.

🤖 Addressed by Claude Code

Resolve orchestrator conflicts by keeping main rate-limit admission and ZDR
redaction while preserving automatic_proxy selection receipts and restored
workflow-run request_id binding; keep both gap-baseline evidence sections.

Co-authored-by: Cursor <cursoragent@cursor.com>
@seonghobae
seonghobae changed the base branch from autoresearch/20260909-kpi-loop to main September 17, 2026 12:49
@seonghobae
seonghobae marked this pull request as ready for review September 17, 2026 13:09
@seonghobae
seonghobae merged commit 1e24037 into main Sep 17, 2026
20 of 25 checks passed
@seonghobae
seonghobae deleted the codex/request-outcome-export-integrated-20260912 branch September 17, 2026 13:22
seonghobae added a commit that referenced this pull request Sep 17, 2026
Preserve cancellable race attempts with decision attempt IDs, route
embeddings through _available_embedding_agents, and keep the PR #971
timeout-supersession narrative alongside main's diary entries.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment on lines +4774 to +4780
phases = self._conn.execute(
"SELECT kind, key, payload FROM orchestration_records "
"WHERE kind IN ('initial_decision', 'decision_receipt') "
"AND key IN (" + placeholders + ") "
"ORDER BY seq DESC LIMIT ?",
(*request_ids, 2 * limit + 1),
).fetchall()
Comment on lines +4781 to +4786
diagnostics = self._conn.execute(
"SELECT kind, key, payload FROM orchestration_records "
"WHERE kind IN ('provider_dispatch', 'auxiliary_dispatch') "
"AND key IN (" + placeholders + ") ORDER BY seq DESC LIMIT ?",
(*request_ids, 8 * limit + 1),
).fetchall()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request priority: medium Normal-priority or P2 work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants