Skip to content

feat(research): add Zotero classification audit evidence - #11

Draft
seonghobae wants to merge 29 commits into
feat/zotero-golden-set-evaluationfrom
autoresearch/zotero-reclassification-sep04
Draft

feat(research): add Zotero classification audit evidence#11
seonghobae wants to merge 29 commits into
feat/zotero-golden-set-evaluationfrom
autoresearch/zotero-reclassification-sep04

Conversation

@seonghobae

@seonghobae seonghobae commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Current source-scope and derived audit repair — 2026-09-06

Exact head 6dff8c2ee42cfeb7bf8688c1f7e95989b61be266 normally preserves prior #11 1dc032598b41a35d52c09d8690c871e07365d7e3 and parent #10 fdf8b8d70c05bcb76c55cb6336c9bf31b5e42ce4. Base remains feat/zotero-golden-set-evaluation; OPEN Draft. Runtime is 935e035, final test delta 23178a9.

RED ebdd852 reproduced findings 3934799129 and 3934994550: forged audit reaches governance, and duplicate parent/child coordinates count as provenance-complete. Source digest still represents source, not counters. The selected alternative is shared deterministic audit recomputation before governance; unique nonblank parent/linked-child keys are required for complete provenance. The earlier reply rejecting source-digest mixing does not remove the need for this derived-audit validation. Unknown original-source authenticity and duplicate proposal authorization are not claimed.

PR10 source partition validation, inventory, pending ancestry and v2 approval binding remain intact. Original prediction-mismatch and independent-approval tests now use coherent attacker-controlled counts and retain their rejection expectations. Changing duplicate candidates together with their count remains a subsequent duplicate-owner binding Gap, not authority granted here.

Exact-head locked workspace tests75/15unfiltered suites including2doctests; runtime strictClippy, warnings-denied rustdoc, release build, format, CI contract and diff checks passed. Independent read-only source review reran17integritytests and found no new regression; this is not GitHub approval. Unchanged pinned coverage gate passed140/140functions1101/1101normalizedregions182/182normalizedbranches. Raw1502/1505lines2332/2343regions177/182branches are below100%.

Fresh native Zotero screenshot plus accessibility state both displayed3719items with list rows and attachment icons. Previous Mac-lock limitation is no longer current. Private bibliographic content/screenshots are not committed; this is only visual observation, not completed reclassification. Actual decisions/approvals/writes remain unchanged. Hosted exact-head checks, independent approval, protected integration, release and later restoration/review/write adoption remain pending. Earlier checkpoints below are historical.

Latest bounded-read integration checkpoint

Exact head 1dc032598b41a35d52c09d8690c871e07365d7e3 normally merges parent 4bb633305b04a1dd4c4ce526806c9469bcb79fd3 while retaining previous child 11d158b105cbd03edc34452358ebb3ff445e388e. Base remains feat/zotero-golden-set-evaluation. The #9 whole-snapshot elapsed-time repair and its RED/GREEN evidence are inherited without reverse-merging later features or discarding predecessor deltas.

This exact head passes Rust 1.98.0 locked workspace tests=60 suites=15, including doctests and excluding filtered subprocess duplicates, strict all-target Clippy, warnings-denied rustdoc, formatting, the existing CI contract and diff checks. Log: /private/tmp/conceptweave-deadline-pr11-20260906.log. Intermediate coverage is not inferred from owner/final-endpoint coverage. No new dependency, actual paper read/decision, Zotero mutation or authority issuer was used.

Draft and protected prerequisites remain. Local tests are not hosted GREEN, independent approval, merged/released source or evidence for another head. Earlier checkpoints below are retained history, not the current head.

Current source-integrity note — 2026-09-05

  • Exact head: autoresearch/zotero-reclassification-sep04@082710e8c9a0e37ddae528e305cd75ab3574d0bb.
  • Exact base: feat/zotero-golden-set-evaluation@e7d4e59f1b55b5954c5f8436527bc96e7ef2fb13.
  • This head inherits PR feat(research): add steward golden-set evaluation #10 root e7d4e59f1b55b5954c5f8436527bc96e7ef2fb13 through ordinary merge ancestry. The source-snapshot digest binds complete captured raw provider JSON and the actual typed classifier inputs; source evidence and derived proposals retain separate identities.
  • GoldenSetApproval.proposal_digest is required and binds the complete proposal records used for evaluation. The current proposal digest is checked before the caller-owned governance verifier. Do not backfill old receipts: regenerate evidence and obtain a new approval bound to the reviewed evidence.
  • Keep Draft. This note does not claim current exact-head hosted GREEN, independent approval, protected merge, live Zotero mutation, or governed publication. Root, predecessor, and terminal-stack local test evidence is not transferred as per-PR hosted evidence.

Earlier heads, runtime snapshots, campaign counts, and verification statements below are historical notes, not current acceptance evidence.

Historical PR notes — original text retained

Outcome

Adds aggregate classification-audit evidence on top of PR #10: snapshot count, bibliographic count, proposal count, provenance-complete count, abstentions, duplicate candidates, zero-success-path failures, and per-disposition totals. The report stays local and aggregate; it does not promote classifier output or Zotero metadata to governed semantic truth.

Exact stack — 2026-09-05

Current review disposition

Two prior current-line findings were checked against the actual contracts rather than accepted mechanically.

The P1 request to include audit_summary in snapshot_digest is not a valid repair for this boundary. ClassificationReport::snapshot_digest and GoldenSetApproval::snapshot_digest are immutable Zotero source-snapshot content identities; TRD §11 states that the digest covers every raw Zotero item in canonical key order. audit_summary is derived aggregate evidence from that same in-memory snapshot and is not used as semantic authority by golden-set evaluation. A future published report/artifact may define its own artifact digest at that publication boundary without redefining source-snapshot identity.

The P2 provenance-completeness finding remains valid. Test-only predecessor b836db910a07cb3c3fc3fb68140d70bcd8d6c6c2 added a parent+linked-note regression requiring a proposal with a blank linked-child Zotero key to be excluded from provenance_complete_count; it did not execute before concurrent source repair. The retained production predicate requires the parent key and every retained child_item_key to be nonblank. The P2 thread remains unresolved until the restacked exact head receives terminal verification; source repair alone is not described as RED→GREEN.

Audit semantics

ClassificationAudit is aggregate-only. A successful report records snapshot/bibliographic/proposal/provenance/abstention/duplicate/disposition totals and zero reported failures; reader errors fail before a successful report is returned. Zotero 9 item version zero remains a valid observed revision coordinate rather than missing provenance. Provenance completeness additionally requires stable identity for every linked child coordinate retained by the proposal.

Inherited PR #10 boundary

PR #10 remains Foundation-dependent. Its current head retains complete observed snapshot identity, raw-snapshot SHA-256 content binding, duplicate-key rejection independent of version, and caller-owned verification of the complete ReviewedGoldenSet. #11 must not become independently merge-ready ahead of #10 or bypass any exact-head finding in its parent.

Zotero boundary

No Zotero mutation exists in this stack. Zotero remains the bibliographic system of record; write-back is a separate Zotero 10+ reviewed capability with authorization, server identity, fresh preconditions, before/after receipts and rollback. Aggregate local counts are evaluation evidence only, never published semantic truth.

Next causal step

Keep Draft. Require exact cb365fb... to execute Product/coverage/rustdoc on one unchanged head. If it fails, repair only the causal failure. Resolve the remaining P2 review thread only after current-head verification. PR #12 and later descendants must retain this current parent ancestry before their evidence is treated as current-stack evidence.

Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-04T14:43:56.265815Z c28fcca Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

# Conflicts:
#	docs/PRD.md
#	docs/TRD.md
#	docs/adr/0006-zotero-research-intake.md
#	docs/product-technical-gap-baseline.md
@seonghobae
seonghobae marked this pull request as draft September 4, 2026 13:53
Preserve PR #11 audit evidence while adopting the current PR #10 test-only snapshot-content identity contract without rewriting history.

Signed-off-by: Seongho Bae <me@seonghobae.me>

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current-head finding: ClassificationAudit::provenance_complete_count treats item_version == 0 as missing provenance. That is not valid for the supported Zotero 9 Local API path. Zotero's primary Local API documentation says earlier versions reported synced versions, with 0 for objects that had never been synced; local edits also did not necessarily change those versions. Version zero is therefore a legitimate provenance coordinate, not evidence absence. The current regression explicitly expects a zero-version item to be incomplete, so the wrong invariant is executable. Add a test-first correction requiring a nonblank key with version 0 to remain provenance-complete, then make the minimum production/test repair; immutable content identity remains the separate PR #10 snapshot-digest lane.

@chatgpt-codex-connector

Copy link
Copy Markdown

💡 Codex Review

if !verify_approval(&golden.approval) {

P1 Badge Bind the approval to the reviewed labels

With a valid receipt and matching snapshot, changing any golden.labels entry still passes verification because verify_approval receives only GoldenSetApproval, and none of that authenticated data commits to the labels. The function then counts altered values as steward-approved truth, allowing golden-set metrics to be corrupted while reported as verified; bind a canonical label digest into the approval or have the verifier authenticate the complete ReviewedGoldenSet.

AGENTS.md reference: AGENTS.md:L18-L18


let report_snapshot = report
.classified_items
.iter()
.map(|item| SnapshotItemRevision {
item_key: item.item_key.clone(),
item_version: item.item_version,
})

P1 Badge Bind the approval to the entire classifier input

For snapshots containing child notes, attachments, annotations, or other excluded records, this constructs report_snapshot only from top-level bibliographic classified_items, even though those omitted records are part of the classifier input and can affect linked-child evidence. An approval that correctly lists the complete Zotero item-key/version snapshot therefore fails with SnapshotMismatch; omitting those records makes it pass but leaves part of the source snapshot unauthenticated. Retain and compare the full input item-revision identity rather than the classified subset.

AGENTS.md reference: AGENTS.md:L18-L18


if report_snapshot.len() != report.classified_items.len()
|| approved_snapshot.len() != golden.approval.snapshot_items.len()
{

P2 Badge Reject duplicate keys even when versions differ

When a report contains the same Zotero key twice with different item versions, both SnapshotItemRevision pairs remain distinct, so this length check passes and a matching approval is accepted. The later BTreeMap keyed only by item_key silently discards one proposal, producing metrics from an ambiguous report instead of failing closed as duplicate report identities should; validate item-key uniqueness independently of version.

AGENTS.md reference: AGENTS.md:L18-L18

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

# Conflicts:
#	docs/PRD.md
#	docs/TRD.md
#	docs/adr/0006-zotero-research-intake.md
Signed-off-by: Seongho Bae <me@seonghobae.me>
# Conflicts:
#	docs/PRD.md
#	docs/TRD.md
#	docs/adr/0006-zotero-research-intake.md
@seonghobae
seonghobae marked this pull request as ready for review September 4, 2026 14:01
@seonghobae

Copy link
Copy Markdown
Contributor Author

@codex review

Preserve PR #11 audit evidence while non-force adopting the current PR #10 test-only integrity contracts.

Signed-off-by: Seongho Bae <me@seonghobae.me>
@seonghobae
seonghobae marked this pull request as draft September 4, 2026 14:05
Signed-off-by: Seongho Bae <me@seonghobae.me>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f509ebfa36

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/conceptweave-zotero/src/lib.rs
# Conflicts:
#	docs/TRD.md
#	docs/adr/0006-zotero-research-intake.md
@seonghobae
seonghobae marked this pull request as ready for review September 4, 2026 14:09
@seonghobae

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 551e6cb1f7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +709 to +712
provenance_complete_count: classified_items
.iter()
.filter(|item| !item.item_key.trim().is_empty())
.count(),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include linked child identities in the provenance count

When a bibliographic item has a linked child with a blank key, child_index retains that child in the proposal, but this predicate examines only the parent's key and counts the proposal as provenance-complete. Since the linked child can no longer be identified by a stable source coordinate, the audit overstates provenance completeness for malformed/untrusted snapshots; validate the proposal's linked child revisions as well as its own key before incrementing this count.

AGENTS.md reference: AGENTS.md:L18-L18

Useful? React with 👍 / 👎.

@seonghobae
seonghobae marked this pull request as draft September 4, 2026 14:21
@seonghobae
seonghobae marked this pull request as ready for review September 4, 2026 14:26
Non-force two-parent restack onto current golden-set evaluation parent. Preserve the classification-audit delta while inheriting the current Foundation CI contract through #9 and #10.

Signed-off-by: Seongho Bae <me@seonghobae.me>

Copy link
Copy Markdown
Contributor Author

Current-stack correction (2026-09-05): exact base #10 bf35f97a1d5d2538e3370cf70a70b4294c1faed6; exact head 472dd348e8f0f07b154af2f63e868104e7519de5. Non-force restack preserves classification-audit semantics; older body SHAs are historical and do not transfer GREEN.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant