Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .agent/demos/d1-receipts/NON-CLAIMS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Non-claims

- This packet does not prove every sentence in PR #3282's body is independently verified -- only the four claims explicitly checked in report.md are scored; claim 4 is explicitly scored `not_supported`.
- This packet does not establish that `session_refs` correctly resolves every PR reference archive-wide -- only that it resolves this one case with structural evidence.
- This packet does not reproduce on the public seed corpus (seed 1843); it requires read-only access to the live archive and the `Sinity/polylogue` GitHub history.
67 changes: 67 additions & 0 deletions .agent/demos/d1-receipts/PROMPT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# D1 "The Receipts": Claim-vs-Evidence on a Real Merged PR

Predeclaration receipt: `artifact:d1-receipts-predeclaration`.

Pick a real merged, agent-authored PR from this repository. Resolve it to
its authoring/dispatch session **structurally** — via `session_refs`
(kind=`pull_request`), not by regex-scanning message prose or a time-window
heuristic. Then check specific sentences from the PR body against that
session's own recorded tool_use/tool_result blocks: does the evidence
actually support the claim, or is the claim resting on the PR body's own
prose with nothing underneath it?

Product primitives only: `session_refs` (the typed evidence table wired by
PR #3425/#3431), `polylogue read --view correlation`, and structural SQL
reads over `blocks`/`session_refs` for citation (mirroring the exact
read-only query style PR #3392 and PR #3282 themselves used in their own
Verification sections — this demo does not invent a new access pattern).

## Steps

1. Resolve PR → session structurally:
```sql
SELECT session_id, repo, ref_number, url
FROM session_refs
WHERE kind = 'pull_request' AND repo = 'Sinity/polylogue' AND ref_number = 3282;
```
Comment on lines +21 to +26

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Add blank lines around the fenced SQL block.

Markdownlint reports MD031 for this fence. Add a blank line before and after the fence.

Proposed fix
 1. Resolve PR → session structurally:
+
    ```sql
    SELECT session_id, repo, ref_number, url
    FROM session_refs
    WHERE kind = 'pull_request' AND repo = 'Sinity/polylogue' AND ref_number = 3282;
    ```
+
    Cross-check the same resolution through the CLI's own read surface:
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
1. Resolve PR → session structurally:
```sql
SELECT session_id, repo, ref_number, url
FROM session_refs
WHERE kind = 'pull_request' AND repo = 'Sinity/polylogue' AND ref_number = 3282;
```
1. Resolve PR → session structurally:
🧰 Tools
🪛 markdownlint-cli2 (0.23.1)

[warning] 22-22: Fenced code blocks should be surrounded by blank lines

(MD031, blanks-around-fences)


[warning] 26-26: Fenced code blocks should be surrounded by blank lines

(MD031, blanks-around-fences)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.agent/demos/d1-receipts/PROMPT.md around lines 21 - 26, Update the Markdown
around the fenced SQL block in the “Resolve PR → session structurally” section
by adding a blank line immediately before the opening fence and immediately
after the closing fence, preserving the SQL content and surrounding
instructions.

Source: Linters/SAST tools

Cross-check the same resolution through the CLI's own read surface:
`polylogue find "id:<session_id>" then read --view correlation --format json`
(this is the surface PR #3425/#3431 wired `session_refs` into —
`insights/session_commit.py:build_correlation_result` and
`insights/correlation_view.py`).

2. Fetch the PR body from GitHub (`gh pr view 3282 --json body`) and pull
out individually falsifiable sentences — not the whole prose block, each
claim on its own.

3. For each claim, search the resolved session's own `blocks` rows
(`tool_use`/`tool_result`, joined by `tool_id`) for structural evidence:
an exact command, an exact exit code, an exact pytest summary line. A
claim with no matching block is marked **not independently verified in
this session** — never silently upgraded to "supported" because the PR
body asserts it.

4. Render the two columns: claimed sentence | observed block evidence
(drillable via the cited `block:` ref), with an explicit status per row.

## Note on this run

This demo's session turned out to be a **merge-conductor** session: its own
`blocks` are almost entirely `Bash` (53 of 56 tool_use blocks) plus 3 `Read`
calls — zero `Edit`/`Write` tool_use. The actual file edits for PR #3282
happened in separately dispatched worker sessions across multiple git
worktrees (`/realm/worktrees/polylogue-membership-head*`); this session
orchestrates `git`, `gh pr create`, and `devtools test`/`devtools verify`
invocations across those worktrees and stitches the result into one PR.

This is itself a real, useful finding, not a inconvenience to hide: the PR

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Fix the article before “inconvenience.”

Use “an inconvenience” because inconvenience starts with a vowel sound.

Proposed fix
-This is itself a real, useful finding, not a inconvenience to hide:
+This is itself a real, useful finding, not an inconvenience to hide:
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
This is itself a real, useful finding, not a inconvenience to hide: the PR
This is itself a real, useful finding, not an inconvenience to hide: the PR
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.agent/demos/d1-receipts/PROMPT.md at line 57, Update the sentence in
PROMPT.md so the phrase before “inconvenience” uses the article “an” rather than
“a,” preserving the rest of the wording.

body's own Verification section names a 7-file `devtools test` invocation
("`devtools test tests/unit/sources/test_live_batch_support.py ...` — all
passing, **see individual commit messages for per-commit pass counts**") —
its own parenthetical admits the aggregate command was never run as one
shot. Searching this session's blocks confirms it: the 7-file string only
appears inside the `gh pr create --body` tool_input (i.e. inside the PR body
text itself), never as an actual invoked command. That specific claim is
marked **not independently verified in this session** in `report.md` and
`checks.json` — precisely the honesty discipline this packet exists to
enforce, applied to itself.
7 changes: 7 additions & 0 deletions .agent/demos/d1-receipts/checks.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 11 additions & 0 deletions .agent/demos/d1-receipts/evidence.ndjson
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{"ref": "artifact:d1-receipts-evidence", "cited_for": "Demo Packet v2 receipt root", "verified_via": "committed evidence.ndjson"}
{"ref": "session:claude-code-session:5ecdb160-495a-4d9b-b80a-3a24886af8cc", "cited_for": "session_refs kind=pull_request resolves this session to Sinity/polylogue#3282", "verified_via": "sqlite3 index.db: SELECT session_id,repo,ref_number,url FROM session_refs WHERE kind='pull_request' AND repo='Sinity/polylogue' AND ref_number=3282"}
{"ref": "block:claude-code-session:5ecdb160-495a-4d9b-b80a-3a24886af8cc:ae9a4788-4bf6-4b89-b01e-90f10e622981:0", "cited_for": "the gh pr create tool_use --body text is byte-identical to the PR body later fetched live via gh pr view 3282, including the exact bullet claims checked below", "verified_via": "diff of tool_input command against `gh pr view 3282 --repo Sinity/polylogue --json body`"}
{"ref": "block:claude-code-session:5ecdb160-495a-4d9b-b80a-3a24886af8cc:a98d94d0-831f-4cf6-946b-40bf99179283:0", "cited_for": "gh pr create tool_result: https://github.com/Sinity/polylogue/pull/3282 -- confirms this session actually opened PR #3282, not merely referenced it", "verified_via": "tool_result text (single URL line)"}
{"ref": "block:claude-code-session:5ecdb160-495a-4d9b-b80a-3a24886af8cc:bdacfb31-ecc7-4491-be2f-891f8bfb888b:0", "cited_for": "tool_use invoking `timeout 180 devtools verify --quick`, checking the PR claim 'devtools verify --quick -- pass'", "verified_via": "tool_input command text"}
{"ref": "block:claude-code-session:5ecdb160-495a-4d9b-b80a-3a24886af8cc:9786b513-9ad0-4e25-ad3e-e18a0f60220f:0", "cited_for": "tool_result: structured verify-run JSON, every step exit=0, total_duration_s=32.99, exit_code=0 -- SUPPORTS the claim via structure, not a trusted pass/fail word in prose", "verified_via": "tool_result JSON body (per-step name/duration_s/exit array)"}
{"ref": "block:claude-code-session:5ecdb160-495a-4d9b-b80a-3a24886af8cc:ebe785a2-de91-4de3-9b28-43bd1a8e9596:0", "cited_for": "tool_use invoking devtools test against tests/unit/maintenance/test_rebuild_index_bulk_build.py (+4 more files), checking the Solution-section claim 'rebuild_index bulk FTS materialization checkpoints progress'", "verified_via": "tool_input command text"}
{"ref": "block:claude-code-session:5ecdb160-495a-4d9b-b80a-3a24886af8cc:59a1be2d-7881-4d64-997d-d055dd61aa74:0", "cited_for": "tool_result: pytest summary '123 passed in 8.05s', ok (12.2s) -- SUPPORTS the rebuild_index claim", "verified_via": "tool_result text (pytest summary line)"}
{"ref": "polylogue-6mvg", "cited_for": "the PR body's own tracking-item reference ('Ref polylogue-6mvg'), cited verbatim inside the gh pr create --body text", "verified_via": "block:...ae9a4788... tool_input"}
{"ref": "action.tool_use.count=53,Bash", "cited_for": "56 total tool_use blocks in this session: 53 Bash, 3 Read, 0 Edit, 0 Write -- this is a merge-conductor/orchestration session, not the direct file-editing session", "verified_via": "sqlite3 index.db: SELECT tool_name, count(*) FROM blocks WHERE session_id=... AND block_type='tool_use' GROUP BY tool_name"}
{"ref": "action.negative-control.missing-7-file-devtools-test-invocation", "cited_for": "counterexample: the PR body's exact 7-file `devtools test tests/unit/sources/test_live_batch_support.py ...` string appears ONLY inside the gh-pr-create --body text (i.e. inside the PR body itself), never as an actually-invoked command in this session's blocks -- 0 matching tool_use rows when the gh-pr-create block is excluded", "verified_via": "sqlite3 index.db: SELECT count(*) FROM blocks WHERE session_id=... AND block_type='tool_use' AND tool_input LIKE '%test_live_batch_support.py%' AND tool_input NOT LIKE '%gh pr create%' -- returns 0"}
6 changes: 6 additions & 0 deletions .agent/demos/d1-receipts/finding.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
archive_cursor: "live-archive:/realm/db/polylogue (read-only, file:...?mode=ro)"
measure_version: demo-packet-v2
commit_sha: 59744a30bf461a587cc679ee62a432e8cd2cf82a
sample_frame_predicate: "session_refs WHERE kind='pull_request' AND repo='Sinity/polylogue' AND ref_number=3282, resolving to claude-code-session:5ecdb160-495a-4d9b-b80a-3a24886af8cc"
run_date: "2026-07-31"
claim: "session_refs typed pull_request evidence resolves a real merged PR to its authoring/dispatch session, and specific PR-body verification sentences can be checked against that session's own recorded blocks -- with unsupported claims marked as such, not silently believed"
135 changes: 135 additions & 0 deletions .agent/demos/d1-receipts/packet.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions .agent/demos/d1-receipts/queries.ndjson
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{"text": "SELECT session_id, repo, ref_number, url FROM session_refs WHERE kind='pull_request' AND repo='Sinity/polylogue' AND ref_number=3282", "lowered_spec": {"unit": "session_ref", "predicate_kind": "structural_equality", "table": "session_refs"}}
{"text": "find \"id:claude-code-session:5ecdb160-495a-4d9b-b80a-3a24886af8cc\" then read --view correlation --format json", "lowered_spec": {"unit": "session", "entry": "id", "view": "correlation"}}
{"text": "SELECT tool_name, count(*) FROM blocks WHERE session_id='claude-code-session:5ecdb160-495a-4d9b-b80a-3a24886af8cc' AND block_type='tool_use' GROUP BY tool_name ORDER BY 2 DESC", "lowered_spec": {"unit": "block", "pipeline_stages": ["group:tool_name", "count"]}}
{"text": "SELECT tu.block_id, tr.text FROM blocks tu JOIN blocks tr ON tr.tool_id=tu.tool_id AND tr.block_type='tool_result' AND tr.session_id=tu.session_id WHERE tu.session_id='claude-code-session:5ecdb160-495a-4d9b-b80a-3a24886af8cc' AND tu.tool_input LIKE '%devtools verify --quick%'", "lowered_spec": {"unit": "block", "predicate_kind": "join_tool_use_to_tool_result"}}
{"text": "SELECT count(*) FROM blocks WHERE session_id='claude-code-session:5ecdb160-495a-4d9b-b80a-3a24886af8cc' AND block_type='tool_use' AND tool_input LIKE '%test_live_batch_support.py%' AND tool_input NOT LIKE '%gh pr create%'", "lowered_spec": {"unit": "block", "predicate_kind": "negative_control_count", "expected_result": 0}}
Loading