Skip to content
Merged
2 changes: 1 addition & 1 deletion docs/data-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Convenience properties resolve these:

| Field | Type | Description |
|-------|------|-------------|
| `id` | `str` | Message ID, `session_id:native_id` (or `session_id:position.variant`) |
| `id` | `str` | Message ID, `session_id:n:native_id` (or `session_id:p:position.variant`) |
| `role` | `Role` | `user`, `assistant`, `system`, `tool`, `unknown` |
| `text` | `str?` | Flattened message text |
| `timestamp` | `datetime?` | Message timestamp |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,21 @@ archive: <demo-archive>
verdict: contradicted_at_claim_time_then_repaired

assistant claim: All tests pass. The clock fix is complete.
claim evidence: block:codex-session:demo-receipts:receipts-a-claim:0
claim evidence: block:codex-session:demo-receipts:n:receipts-a-claim:0

at claim time:
tool: shell (exec_command)
command: pytest tests/test_clock.py -q
exit: 1 (failed=true)
result: {"metadata": {"exit_code": 1}, "output": "F tests/test_clock.py::test_uses_monotonic_clock\n1 failed in 0.18s"}
evidence: block:codex-session:demo-receipts:call-receipts-test-fail:0
evidence: block:codex-session:demo-receipts:n:call-receipts-test-fail:0

later recovery:
tool: shell (exec_command)
command: pytest tests/test_clock.py -q
exit: 0 (failed=false)
result: {"metadata": {"exit_code": 0}, "output": ". 1 passed in 0.16s"}
evidence: block:codex-session:demo-receipts:call-receipts-test-pass:0
evidence: block:codex-session:demo-receipts:n:call-receipts-test-pass:0

anti-grep control:
prose hits for 'error': 2
Expand All @@ -29,7 +29,7 @@ source material:
blob_sha256: 9fd0dbdb080058070935924534a903cc63a8dcba571f6b2734f92a96576b59d7

completion-claim experiment:
sample manifest: ffbb3c4609d488d25510923848188422bb098ef700fd862e44e4315e14b473f6
sample manifest: a7a39bdcced3d950886d5b555d2ceec7b5f776d961fa726a76946c857f863d13
denominator: 2
unsupported by structural evidence: 0 (0.0%)
neutral prior outcome: 0 (0.0%)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,25 +18,25 @@

Map the demo lineage base context.

`codex-session:demo-lineage-parent:parent-u0`
`codex-session:demo-lineage-parent:n:parent-u0`

### 2026-07-04T10:00:02+00:00 - assistant / message

I have the base context and can branch the analysis.

`codex-session:demo-lineage-parent:parent-a1`
`codex-session:demo-lineage-parent:n:parent-a1`

### 2026-07-04T10:01:03+00:00 - user / message

Now take the forked branch and audit construct validity.

`codex-session:demo-lineage-fork:fork-u2`
`codex-session:demo-lineage-fork:n:fork-u2`

### 2026-07-04T10:01:04+00:00 - assistant / message

The fork diverges into demo corpus construct checks.

`codex-session:demo-lineage-fork:fork-a3`
`codex-session:demo-lineage-fork:n:fork-a3`

### Last Messages

Expand Down
16 changes: 8 additions & 8 deletions docs/examples/demo-tour/report.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
"result": "pass",
"triggered": false
},
"first_result_s": 2.065,
"first_result_s": 1.362,
"non_claims": [
"The deterministic tour does not establish field prevalence, production scale, or provider completeness.",
"The deterministic tour does not establish memory uplift, invoice accuracy, selective deletion, or the Sinex backend.",
Expand Down Expand Up @@ -379,13 +379,13 @@
},
"steps": [
{
"bytes_written": 1399,
"bytes_written": 1405,
"command": [
"polylogue",
"demo",
"receipts"
],
"duration_s": 2.064,
"duration_s": 1.362,
"exit_code": 0,
"name": "claim versus receipt",
"output_path": "command-output/01-claim-versus-receipt.txt"
Expand All @@ -396,13 +396,13 @@
"polylogue",
"actions where is_error:true | group by tool | count"
],
"duration_s": 2.822,
"duration_s": 1.839,
"exit_code": 0,
"name": "failed actions aggregate",
"output_path": "command-output/02-failed-actions-aggregate.txt"
},
{
"bytes_written": 936,
"bytes_written": 944,
"command": [
"polylogue",
"--id",
Expand All @@ -411,7 +411,7 @@
"--view",
"chronicle"
],
"duration_s": 2.485,
"duration_s": 1.695,
"exit_code": 0,
"name": "composed lineage",
"output_path": "command-output/03-composed-lineage.txt"
Expand All @@ -423,13 +423,13 @@
"analyze",
"--facets"
],
"duration_s": 2.435,
"duration_s": 1.629,
"exit_code": 0,
"name": "archive facets",
"output_path": "command-output/04-archive-facets.txt"
}
],
"total_duration_s": 15.747,
"total_duration_s": 27.096,
"transcript_path": "transcript.txt",
"verify": {
"absolute_path_leaks": [],
Expand Down
12 changes: 6 additions & 6 deletions docs/examples/demo-tour/report.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ The semantic fixture verifier runs before the narrated commands and checks plant

## Timings

- First evidence result: 2.065s (budget 30s)
- Full tour: 15.747s (budget 420s)
- First evidence result: 1.362s (budget 30s)
- Full tour: 27.096s (budget 420s)

## Archive

Expand All @@ -55,10 +55,10 @@ The semantic fixture verifier runs before the narrated commands and checks plant

| Step | Exit | Duration | Bytes | Output |
| --- | ---: | ---: | ---: | --- |
| claim versus receipt | 0 | 2.064s | 1399 | `command-output/01-claim-versus-receipt.txt` |
| failed actions aggregate | 0 | 2.822s | 62 | `command-output/02-failed-actions-aggregate.txt` |
| composed lineage | 0 | 2.485s | 936 | `command-output/03-composed-lineage.txt` |
| archive facets | 0 | 2.435s | 1652 | `command-output/04-archive-facets.txt` |
| claim versus receipt | 0 | 1.362s | 1405 | `command-output/01-claim-versus-receipt.txt` |
| failed actions aggregate | 0 | 1.839s | 62 | `command-output/02-failed-actions-aggregate.txt` |
| composed lineage | 0 | 1.695s | 944 | `command-output/03-composed-lineage.txt` |
| archive facets | 0 | 1.629s | 1652 | `command-output/04-archive-facets.txt` |

## Problems

Expand Down
28 changes: 14 additions & 14 deletions docs/examples/demo-tour/transcript.txt
Original file line number Diff line number Diff line change
@@ -1,32 +1,32 @@
# prepare deterministic proof archive
Seeded 19 sessions, 71 messages, and 8 user-state assertions in 12.070s. Fixture audit: 40/40 declared constructs satisfied.
Seeded 19 sessions, 71 messages, and 8 user-state assertions in 20.556s. Fixture audit: 40/40 declared constructs satisfied.

# verify evidence before presenting it
Verification passed in 0.017s; 0 path leaks and 0 semantic problems. The complete fixture and verification audit remains in report.json.
Verification passed in 0.013s; 0 path leaks and 0 semantic problems. The complete fixture and verification audit remains in report.json.

$ polylogue demo receipts
Start with a falsifiable disagreement: assistant prose claims the tests pass, while the provider-normalized tool result says exit 1. A later run repairs the result, and a prose-only 'error' control demonstrates why keyword matching is not the oracle.
exit=0 duration=2.267s bytes=1399
exit=0 duration=1.362s bytes=1405
Polylogue evidence receipt
archive: <demo-archive>
verdict: contradicted_at_claim_time_then_repaired

assistant claim: All tests pass. The clock fix is complete.
claim evidence: block:codex-session:demo-receipts:receipts-a-claim:0
claim evidence: block:codex-session:demo-receipts:n:receipts-a-claim:0

at claim time:
tool: shell (exec_command)
command: pytest tests/test_clock.py -q
exit: 1 (failed=true)
result: {"metadata": {"exit_code": 1}, "output": "F tests/test_clock.py::test_uses_monotonic_clock\n1 failed in 0.18s"}
evidence: block:codex-session:demo-receipts:call-receipts-test-fail:0
evidence: block:codex-session:demo-receipts:n:call-receipts-test-fail:0

later recovery:
tool: shell (exec_command)
command: pytest tests/test_clock.py -q
exit: 0 (failed=false)
result: {"metadata": {"exit_code": 0}, "output": ". 1 passed in 0.16s"}
evidence: block:codex-session:demo-receipts:call-receipts-test-pass:0
evidence: block:codex-session:demo-receipts:n:call-receipts-test-pass:0

anti-grep control:
prose hits for 'error': 2
Expand All @@ -38,7 +38,7 @@ source material:
blob_sha256: 9fd0dbdb080058070935924534a903cc63a8dcba571f6b2734f92a96576b59d7

completion-claim experiment:
sample manifest: ffbb3c4609d488d25510923848188422bb098ef700fd862e44e4315e14b473f6
sample manifest: a7a39bdcced3d950886d5b555d2ceec7b5f776d961fa726a76946c857f863d13
denominator: 2
unsupported by structural evidence: 0 (0.0%)
neutral prior outcome: 0 (0.0%)
Expand All @@ -47,14 +47,14 @@ contradicted without recorded repair: 1 (50.0%)

$ polylogue 'actions where is_error:true | group by tool | count'
Now aggregate the same structural field across providers. This query counts normalized failed actions; it does not search prose for the word 'error'.
exit=0 duration=3.273s bytes=62
exit=0 duration=1.839s bytes=62
tool=Bash count=4
tool=exec_command count=2
tool=Edit count=1

$ polylogue --id codex-session:demo-lineage-fork read --view chronicle
Read a fork as one logical chronicle: inherited parent messages remain attributable to their origin while the fork contributes only its divergent tail.
exit=0 duration=2.708s bytes=936
exit=0 duration=1.695s bytes=944
# Session Chronicle

- Sessions: 1
Expand All @@ -75,33 +75,33 @@ exit=0 duration=2.708s bytes=936

Map the demo lineage base context.

`codex-session:demo-lineage-parent:parent-u0`
`codex-session:demo-lineage-parent:n:parent-u0`

### 2026-07-04T10:00:02+00:00 - assistant / message

I have the base context and can branch the analysis.

`codex-session:demo-lineage-parent:parent-a1`
`codex-session:demo-lineage-parent:n:parent-a1`

### 2026-07-04T10:01:03+00:00 - user / message

Now take the forked branch and audit construct validity.

`codex-session:demo-lineage-fork:fork-u2`
`codex-session:demo-lineage-fork:n:fork-u2`

### 2026-07-04T10:01:04+00:00 - assistant / message

The fork diverges into demo corpus construct checks.

`codex-session:demo-lineage-fork:fork-a3`
`codex-session:demo-lineage-fork:n:fork-a3`

### Last Messages

_No distinct matching prose in the last edge._

$ polylogue analyze --facets
Only after inspecting evidence, zoom out to the archive across 8 origins, with deferred families labeled rather than silently guessed.
exit=0 duration=2.555s bytes=1652
exit=0 duration=1.629s bytes=1652
Facets (global) — matched result set:
readiness: ready (cost_class=cheap; budget 0.01s/2.00s)
sessions: 19 messages: 71
Expand Down
25 changes: 25 additions & 0 deletions docs/maintenance.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,31 @@ reclassifies under `BEGIN IMMEDIATE` before fsyncing the prepared receipt and
deleting the exact candidate set. Review the receipt's final `committed` line
before treating the pass as complete.

### `polylogue ops maintenance blob-reference-closure` - acquired reference closure

Read-only by default. It checks that each `raw_sessions` row has exactly one
matching `raw_payload` ref and that each acquired index attachment is reachable
through `attachment_refs`. Raw gaps are repaired from the retained raw row's
exact hash, path, size, and acquisition timestamp. Attachment gaps are repaired
only when a complete reparse of authoritative `source.db` bytes reproduces the
attachment identity and its owning message still exists. Other rows are
reported as typed blockers and remain untouched.

```bash
polylogue ops maintenance blob-reference-closure --output-format json
polylogue ops maintenance blob-reference-closure --apply \
--backup-manifest /path/to/verified-full-evidence-manifest.json \
--receipt-file /path/to/new/blob-reference-closure.jsonl \
--output-format json
```

Apply requires the daemon to be offline, a verified backup manifest covering
both `source.db` and `index.db`, and a new receipt path. It inserts exact refs
only, never deletes or replaces existing refs. The source and index commits are
recorded separately in the receipt so a retry can safely continue an additive
repair. Reindex acceptance runs the same closure check against the candidate
index before promotion.

### `polylogue ops maintenance hook-payload-ref-reconcile` - legacy hook-ref repair

Read-only by default. It classifies historical orphaned `raw_payload` refs and
Expand Down
6 changes: 6 additions & 0 deletions polylogue/cli/commands/maintenance/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,12 @@
"blob_reference_liveness_command",
"Classify source-tier orphan refs; apply only with backup and receipt.",
),
(
"blob-reference-closure",
"_blob_reference_closure",
"blob_reference_closure_command",
"Repair deterministic raw and acquired-attachment reference gaps; dry-run by default.",
),
(
"hook-payload-ref-reconcile",
"_hook_payload_ref_reconciliation",
Expand Down
63 changes: 63 additions & 0 deletions polylogue/cli/commands/maintenance/_blob_reference_closure.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
"""CLI adapter for acquired blob-reference closure repair."""

from __future__ import annotations

import json
from pathlib import Path

import click

from polylogue.paths import archive_root


@click.command("blob-reference-closure")
@click.option("--apply", "apply_changes", is_flag=True, help="Apply only deterministic exact reference repairs.")
@click.option(
"--backup-manifest",
type=click.Path(dir_okay=False, path_type=Path),
default=None,
help="Verified manifest covering source.db and index.db; required with --apply.",
)
@click.option(
"--receipt-file",
type=click.Path(dir_okay=False, path_type=Path),
default=None,
help="New immutable receipt path; required with --apply.",
)
@click.option("--output-format", type=click.Choice(["plain", "json"]), default="plain", show_default=True)
def blob_reference_closure_command(
apply_changes: bool,
backup_manifest: Path | None,
receipt_file: Path | None,
output_format: str,
) -> None:
"""Audit closure, or repair exact refs from existing source evidence."""
from polylogue.maintenance.blob_reference_closure import (
BlobReferenceClosureError,
reconcile_blob_reference_closure,
)

try:
report = reconcile_blob_reference_closure(
archive_root(),
backup_manifest=backup_manifest,
receipt_path=receipt_file,
dry_run=not apply_changes,
)
except BlobReferenceClosureError as exc:
raise click.ClickException(str(exc)) from exc

payload = {"mode": "blob_reference_closure", **report.to_dict()}
if output_format == "json":
click.echo(json.dumps(payload, indent=2, sort_keys=True))
return
click.echo("Blob-reference closure")
click.echo(f"Mode: {'apply' if report.applied else 'dry-run'}")
click.echo(f"Raw repair: {report.raw_repaired_count:,}")
click.echo(f"Attachment: {report.attachment_repaired_count:,}")
click.echo(f"Blockers: {len(report.plan.blockers):,}")
for blocker in report.plan.blockers:
click.echo(f" blocker [{blocker.kind.value}] {blocker.object_id}: {blocker.detail}")


__all__ = ["blob_reference_closure_command"]
11 changes: 6 additions & 5 deletions polylogue/core/identity_law.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,13 +43,14 @@ def message_local_id(
) -> str:
"""Return the message-local identity component.

Provider-native message IDs win when present. When the provider omits a
native ID, archive falls back to ``position.variant_index`` so sibling
regeneration branches cannot collide.
Provider-native message IDs and position-derived coordinates occupy
disjoint tagged namespaces. This prevents a provider id such as ``0.0``
from colliding with the positional identity for ``(position=0,
variant_index=0)`` while keeping both components opaque.
"""
if native_id is not None and native_id.strip():
return _required_text("message native_id", native_id)
return f"{_required_non_negative('position', position)}.{_required_non_negative('variant_index', variant_index)}"
return f"n:{_required_text('message native_id', native_id)}"
return f"p:{_required_non_negative('position', position)}.{_required_non_negative('variant_index', variant_index)}"
Comment on lines 51 to +53

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Show the stored-native-id normalizer and every caller of message_local_id.
ast-grep run --pattern 'def _stored_message_native_id($$$) { $$$ }' --lang python polylogue/storage/sqlite/archive_tiers/write.py 2>/dev/null
rg -nP -C6 'def _stored_message_native_id' polylogue/storage/sqlite/archive_tiers/write.py
rg -nP -C3 '\bmessage_local_id\s*\(' --type=py
rg -nP -C3 '\bmessage_id\s*\(' polylogue/core/identity_law.py

Repository: Sinity/polylogue

Length of output: 838


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== files =="
git ls-files | rg '(^|/)polylogue/(core/identity_law\.py|storage/sqlite/archive_tiers/(write\.py|archive_tiers_specs\.py)|core/\w*id|storage/sqlite/archive_tiers/\w*id)'

echo
echo "== identity_law outline and relevant lines =="
ast-grep outline polylogue/core/identity_law.py --view expanded || true
sed -n '1,120p' polylogue/core/identity_law.py

echo
echo "== writer native normalizer relevant lines =="
sed -n '6550,6630p' polylogue/storage/sqlite/archive_tiers/write.py

echo
echo "== message_id generation relevant lines =="
rg -n -C8 'message_id|required.*message_id|n:\||n:' polylogue/storage/sqlite/archive_tiers/archive_tiers_specs.py polylogue/storage/sqlite/archive_tiers/write.py polylogue/core/identity_law.py

echo
echo "== message_local_id call sites =="
rg -n -C4 '\bmessage_local_id\s*\(' --glob '*.py'
rg -n -C4 '\bmessage_id\s*\(' --glob '*.py'

Repository: Sinity/polylogue

Length of output: 50373


🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
import ast, pathlib, re

for path in [
    pathlib.Path("polylogue/core/identity_law.py"),
    pathlib.Path("polylogue/storage/sqlite/archive_tiers/write.py"),
]:
    print(f"\n== {path} syntax check ==")
    ast.parse(path.read_text())
    print("OK")

files = [pathlib.Path(p) for p in pathlib.Path("polylogue").rglob("*.py")]
for sym in ["message_local_id", "message_id"]:
    print(f"\n== {sym} call sites in polylogue Python files ==")
    for p in files:
        data = p.read_text()
        if re.search(rf'\b{sym}\s*\(', data):
            print(p)
PY

echo
echo "== focused identity specs and writer normalizer =="
sed -n '40,70p' polylogue/core/identity_law.py
rg -n -C4 'def _message_id|def message_id|message_id = ' polylogue/storage/sqlite/archive_tiers/write.py polylogue/core/identity_law.py
sed -n '6574,6614p' polylogue/storage/sqlite/archive_tiers/write.py

echo
echo "== focused generated column =="
rg -n -C5 "message_id|n:.*native_id|native_id.*n:" polylogue/storage/sqlite/archive_tiers/archive_tiers_specs.py

Repository: Sinity/polylogue

Length of output: 44447


Normalize native IDs before building message identity strings.

message_local_id() accepts native_id.strip() but embeds the unstripped native_id; this helper only has call sites in message_id(), so normalization can live in message_id()/writers, but archive_tiers_specs.py regenerates message_id from the stored native_id without stripping. Match the Python and generated SQL forms so "n:" + " msg-1 " and "n:" + "msg-1" cannot diverge.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@polylogue/core/identity_law.py` around lines 51 - 53, Normalize native_id by
stripping surrounding whitespace before constructing the native identity in
message_local_id() and the corresponding message_id()/writer paths. Ensure
archive_tiers_specs.py’s regenerated message_id uses the same normalized value,
so Python and generated SQL produce identical IDs for whitespace-padded and
trimmed native IDs.



def message_id(
Expand Down
Loading