feat(research): add steward golden-set evaluation - #10
Conversation
Signed-off-by: Seongho Bae <me@seonghobae.me>
|
@codex review |
|
@coderabbitai review |
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e7d3243fed
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 399794e5ae
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: Seongho Bae <me@seonghobae.me>
|
@codex review |
|
Codex Review: Didn't find any major issues. Can't wait for the next one! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Signed-off-by: Seongho Bae <me@seonghobae.me>
|
@codex review |
…tero-review-integrity-root
# Conflicts: # docs/TRD.md
Preserve current child4bb6333 and verified parent51c7df6. Resolve classifier overlap by retaining raw-source/input snapshot hashing before moving records into inventory. Retain both Proposed ADR amendments; no delta deletion or authority transfer.
Preserve both parent deltas. Validate report partitions before governance, reuse bounded pending traversal, and require v2 approval binding for unclassified metadata. No source writes or approval issuance.
The entry validator already checks unique snapshot and proposal identities. Equal partition count plus successful unique coordinate consumption proves no leftover source. Retain approved receipt checks and all malformed-report regressions; remove no authority boundary.
seonghobae
added
enhancement
ChatGPT Codex Connector
1 participant
Current source-scope admission checkpoint — 2026-09-06
Exact head
fdf8b8d70c05bcb76c55cb6336c9bf31b5e42ce4, runtime8ccb0d5b3d7705786b6c40c3bcf5a10ff32046d9, normally preserves prior #104bb633305b04a1dd4c4ce526806c9469bcb79fd3and parent #951c7df6d03f072449422fd58ca24b2f9d6026f07. Base remainsfeat/zotero-research-classification; OPEN Draft.RED
3f2cf55proved three inventory/approval bypasses. Shared admission now checks complete disjoint source partitions, counts, revisions, types, child links and recomputed unresolved ancestry before governance. Currentconceptweave-classification-proposals-v2supersedes the historical v1 digest below and binds projected unclassified metadata plus pending identities. Original v1 receipts cannot authorize current evaluation. This is not original-source authentication, a full-text backup, approval issuance or Zotero write authority.Exact-head locked workspace: 71 tests/14 unfiltered suites including two doctests. Runtime strict Clippy, warnings-denied rustdoc, release build, format and CI contract passed. Independent read-only source review found no actionable defect and reran 15 integrity tests; this is not GitHub approval. Unchanged pinned coverage gate passed 136/136 functions, 1037/1037 normalized regions, 174/174 normalized branches. Raw lines1448/1451, regions2268/2279, branches169/174 are not100%. No new coverage exclusion or dependency.
PRD/TRD/UML/ProposedADR0006/Gap baseline record the deletion proof for checks subsumed by shared admission. Descendant restoration/worksheet/duplicate/write adoption, actual classification decisions, independent approvals, exact-head hosted checks, protected integration and release remain pending. Fresh visual verification remains unavailable after Mac-lock failure; no new screenshot claim. Earlier checkpoints below are historical, not current source or coverage evidence.
Latest bounded-read integration checkpoint
Exact head
4bb633305b04a1dd4c4ce526806c9469bcb79fd3normally merges parentbb2faccfda9efed55b6759f1bbf7907bf6ec0c3bwhile retaining previous childd4f39b45901ea741baa893a3d6117c5322b7dcdf. Base remainsfeat/zotero-research-classification. The #9 whole-snapshot elapsed-time repair and its RED/GREEN evidence are inherited without reverse-merging later features or discarding predecessor deltas.This exact head passes Rust 1.98.0 locked workspace tests=58 suites=14, including doctests and excluding filtered subprocess duplicates, strict all-target Clippy, warnings-denied rustdoc, formatting, the existing CI contract and diff checks. Log:
/private/tmp/conceptweave-deadline-pr10-20260906.log. Intermediate coverage is not inferred from owner/final-endpoint coverage. No new dependency, actual paper read/decision, Zotero mutation or authority issuer was used.Draft and protected prerequisites remain. Local tests are not hosted GREEN, independent approval, merged/released source or evidence for another head. Earlier checkpoints below are retained history, not the current head.
Outcome
Adds Issue #8's steward-reviewed golden-set evaluation above PR #9. Quality results are aggregate-only; proposals cannot become semantic authority without externally verified review.
Exact stack — 2026-09-05
256076d12dec80997960b1db89bec0809f129c90.e7d4e59f1b55b5954c5f8436527bc96e7ef2fb13.Root integrity repairs
The earlier claim that typed input covered every raw provider field was too strong. Thread 3934854209 exposed omitted provider metadata. Committed RED cases also exposed missing-versus-empty normalization and post-decode input mutation. The final digest now binds complete captured JSON plus actual typed classifier input under
conceptweave-zotero-snapshot-v2, using borrowed pairs rather than cloning a second snapshot. Unknown/nested metadata, field presence and actual input changes all invalidate identity.Thread 3934854221 exposed mutable predictions scored with an unchanged review. Required
GoldenSetApproval.proposal_digestnow binds all complete, sorted proposal records underconceptweave-classification-proposals-v1. Evaluation recomputes that digest and validates local structure/source/proposals/labels before invoking governance. Rewriting the supplied digest cannot reuse an independently verified approval.Existing source-coordinate, linked-child, label-set and duplicate-key protections are retained. PRD, TRD, Proposed ADR 0006, primary-source doctoring and CHANGELOG describe both the root cause and the breaking receipt contract. Old reports require regeneration; old approval JSON fails closed and genuine approvals require reissuance, never automatic backfill.
Executed verification
Runtime and successor boundaries
Zotero 9 revision counters were sync-derived and could miss local edits. The current desktop has independently been read as Zotero 10.0.1/schema 44/library 2; that observation does not transfer old review decisions or grant write authority. No real precision/recall, approval, authorization prompt, write or rollback is claimed here.
The dependent stack is being non-force updated from #11 onward. It must retain the source/proposal bindings, reject legacy receipts during worksheet finalization, test deserialized-report mutation, and preserve strict owner-only artifact and transport boundaries. Keep all dependent PRs Draft while prerequisites and exact-head checks remain outstanding.
Advances #8.