Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
b818c29
test(zotero): require abstention review context
seonghobae Sep 4, 2026
c93a50b
feat(zotero): retain minimal steward review context
seonghobae Sep 4, 2026
1b7ad09
test(zotero): select review fixtures by identity
seonghobae Sep 4, 2026
e5af1b8
docs(zotero): define sensitive steward review context
seonghobae Sep 4, 2026
8ae81e2
style(zotero): format steward review test
seonghobae Sep 4, 2026
a433e2f
docs(zotero): record live steward context coverage
seonghobae Sep 4, 2026
6ea2909
test(zotero): reject duplicate abstract review context
seonghobae Sep 4, 2026
b63d775
fix(zotero): retain abstention abstracts once
seonghobae Sep 4, 2026
3fcd02d
docs(zotero): document single-copy review context
seonghobae Sep 4, 2026
0a606c6
docs(gap): record single-copy Zotero evidence
seonghobae Sep 4, 2026
d7270fe
Merge current reconciliation parent into steward review context
seonghobae Sep 4, 2026
410d93a
Merge repaired write receipt evidence into zotero-steward-review-context
seonghobae Sep 4, 2026
fa860b7
chore(zotero): restack steward review context
seonghobae Sep 4, 2026
d3fbc98
Merge remote-tracking branch 'origin/autoresearch/zotero10-rollback-r…
seonghobae Sep 4, 2026
0312fc5
merge(zotero): adopt current rollback-reconciliation parent
seonghobae Sep 4, 2026
2c8ee56
merge(zotero): adopt current rollback-reconciliation parent and gap b…
seonghobae Sep 5, 2026
5c9f28c
merge(research): inherit verified source and proposal approval binding
seonghobae Sep 5, 2026
10eb8e1
test(research): adopt captured-source review context fixture
seonghobae Sep 5, 2026
e02e520
merge(research): inherit canonical local transport repairs into PR #22
seonghobae Sep 5, 2026
3c14b86
merge(research): inherit deterministic transport framing regression i…
seonghobae Sep 5, 2026
0b0691e
merge(zotero): propagate validated approval ordering through PR 22
seonghobae Sep 5, 2026
7179d13
merge(research): inherit bounded metadata reads into PR #22
seonghobae Sep 6, 2026
49daf4c
merge(research): retain steward context with repaired evidence scope
seonghobae Sep 6, 2026
ebcf2a3
test(zotero): bind steward context changes to approval
seonghobae Sep 6, 2026
b44d603
test(zotero): preserve multilingual context and independent approval
seonghobae Sep 6, 2026
51d1682
docs(research): record private steward context evidence
seonghobae Sep 6, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ All notable changes to ConceptWeave are documented here.
- Draft 2020-12 JSON Schema for the semantic-candidate public contract.
- Standards and research doctoring covering stable W3C ontology standards, 2026 RDF/SHACL work in progress, Apache Ossie, and recent LLM ontology-engineering research.
- Read-only delayed reconciliation receipts for indeterminate Zotero rollback operations.
- Minimal, nonduplicated local abstract context for Zotero items that require steward classification.

### Security

Expand Down
4 changes: 3 additions & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

All source artifacts, generated candidate payloads, external ontology files, model outputs, and future web-retrieved content are untrusted input.

Local Zotero classification reports can contain bibliographic titles, tags, matched values, and abstention abstracts. They are sensitive review material, remain outside the repository, and are not publication artifacts.

## Required controls

- source size, type, nesting, archive/decompression, and parser-time bounds;
Expand Down Expand Up @@ -38,4 +40,4 @@ Zotero 10+ write authorization and mutation use the provider-defined loopback HT

A hostile same-host process capable of binding, observing, or interposing on the loopback endpoint therefore remains an unresolved credential-confidentiality threat. The currently documented Zotero Local API does not provide an HTTPS or OS-authenticated IPC write endpoint that ConceptWeave can substitute. Consequently, mock/local orchestration may be tested, but enterprise-secure live write-back remains fail closed. It may become release-eligible only if Zotero provides a protected transport or an explicit product-security/governance decision narrows the supported threat model and accepts the residual same-host risk. The detailed actor, asset, residual-risk, and release decision is maintained in `THREAT_MODEL.md`, and `docs/TRD.md` carries the same technical boundary.

Security findings become tests before the related runtime capability can be marked release-ready.
Security findings become tests before the related runtime capability can be marked release-ready.
9 changes: 9 additions & 0 deletions crates/conceptweave-zotero/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,9 @@ pub struct ClassifiedItem {
pub item_type: String,
/// Human-readable title retained in the local report only.
pub title: String,
/// Nonempty abstract retained only when a steward must classify the item.
#[serde(skip_serializing_if = "Option::is_none")]
pub review_abstract_note: Option<String>,
/// Collection keys observed with the item.
pub collection_keys: Vec<String>,
/// Tag text observed with the item.
Expand Down Expand Up @@ -2731,11 +2734,17 @@ fn classify_item(item: &ZoteroItem, child_item_keys: Vec<String>) -> ClassifiedI
),
};

let review_abstract_note = (proposed_disposition == Disposition::NeedsStewardReview
&& !item.data.abstract_note.trim().is_empty()
&& !field_values.contains_key("abstract_note"))
.then(|| item.data.abstract_note.clone());

ClassifiedItem {
item_key: item.key.clone(),
item_version: item.version,
item_type: item.data.item_type.clone(),
title: item.data.title.clone(),
review_abstract_note,
collection_keys: item.data.collections.clone(),
tags: item.data.tags.clone(),
proposed_disposition,
Expand Down
171 changes: 171 additions & 0 deletions crates/conceptweave-zotero/tests/steward_review_context.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
use conceptweave_zotero::{
Disposition, EvaluationError, GoldenLabel, GoldenSetApproval, ItemData, ReviewedGoldenSet,
ZoteroItem, classification_proposal_digest, classify_snapshot, evaluate_reviewed_golden_set,
};

fn item(key: &str, title: &str, abstract_note: &str) -> ZoteroItem {
ZoteroItem {
source_record: None,
key: key.into(),
version: 7,
data: ItemData {
item_type: "journalArticle".into(),
title: title.into(),
abstract_note: abstract_note.into(),
doi: String::new(),
parent_item: String::new(),
collections: vec![],
tags: vec![],
},
}
}

#[test]
fn abstentions_retain_only_the_abstract_needed_for_local_steward_review() {
let review_abstract = "A domain-specific vocabulary outside the deterministic rules.";
let report = classify_snapshot(
"9.0.6".into(),
None,
42,
vec![
item("REVIEW01", "Unmatched domain study", review_abstract),
item(
"DECIDED1",
"Ontology learning",
"This abstract must not be copied into review-only context.",
),
item("EMPTY001", "Unmatched title", ""),
item("SPACE001", "Unmatched title", " \n\t "),
item("CJK00001", "Unmatched title", "地域固有の語彙を調査する。"),
item(
"CONFLICT",
"Unmatched title",
"Ontology learning and ontology matching are compared.",
),
],
);

let review = report
.classified_items
.iter()
.find(|item| item.item_key == "REVIEW01")
.unwrap();
assert_eq!(review.proposed_disposition, Disposition::NeedsStewardReview);
assert_eq!(
review.review_abstract_note.as_deref(),
Some(review_abstract)
);

let decided = report
.classified_items
.iter()
.find(|item| item.item_key == "DECIDED1")
.unwrap();
assert_eq!(decided.proposed_disposition, Disposition::Generation);
assert!(decided.review_abstract_note.is_none());

let empty = report
.classified_items
.iter()
.find(|item| item.item_key == "EMPTY001")
.unwrap();
assert_eq!(empty.proposed_disposition, Disposition::NeedsStewardReview);
assert!(empty.review_abstract_note.is_none());

let whitespace = report
.classified_items
.iter()
.find(|item| item.item_key == "SPACE001")
.unwrap();
assert!(whitespace.review_abstract_note.is_none());
assert!(
serde_json::to_value(whitespace)
.unwrap()
.get("review_abstract_note")
.is_none()
);
let non_english = report
.classified_items
.iter()
.find(|item| item.item_key == "CJK00001")
.unwrap();
assert_eq!(
non_english.proposed_disposition,
Disposition::NeedsStewardReview
);
assert_eq!(
non_english.review_abstract_note.as_deref(),
Some("地域固有の語彙を調査する。")
);

let conflict = report
.classified_items
.iter()
.find(|item| item.item_key == "CONFLICT")
.unwrap();
assert_eq!(
conflict.proposed_disposition,
Disposition::NeedsStewardReview
);
assert!(conflict.evidence.field_values.contains_key("abstract_note"));
assert!(conflict.review_abstract_note.is_none());
let serialized = serde_json::to_string(conflict).unwrap();
assert_eq!(serialized.matches("Ontology learning").count(), 1);
}

#[test]
fn changed_review_context_invalidates_prior_approval_before_verification() {
for (original, replacement) in [
(
"Unmatched original abstract.",
Some("Changed review context."),
),
("Unmatched original abstract.", None),
("", Some("Added review context.")),
] {
let mut report = classify_snapshot(
"10.0.1".into(),
Some("synthetic-server".into()),
42,
vec![item("REVIEW01", "Unmatched domain study", original)],
);
let approved = ReviewedGoldenSet {
approval: GoldenSetApproval {
receipt_id: "synthetic-receipt".into(),
reviewer_subject: "synthetic-steward".into(),
library_version: report.library_version,
rule_revision: report.rule_revision.into(),
snapshot_digest: report.snapshot_digest.clone(),
proposal_digest: classification_proposal_digest(&report),
snapshot_items: report.snapshot_items.clone(),
},
labels: vec![GoldenLabel::new("REVIEW01", Disposition::Generation)],
};
let result =
evaluate_reviewed_golden_set(&report, &approved, |set| set == &approved).unwrap();
let aggregate = serde_json::to_string(&result).unwrap();
for private_value in ["REVIEW01", "synthetic-steward", "Unmatched domain study"] {
assert!(!aggregate.contains(private_value));
}
if !original.is_empty() {
assert!(!aggregate.contains(original));
}
report.classified_items[0].review_abstract_note = replacement.map(str::to_owned);
assert_ne!(
classification_proposal_digest(&report),
approved.approval.proposal_digest
);
assert_eq!(
evaluate_reviewed_golden_set(&report, &approved, |_| {
panic!("changed review context must fail before governance")
}),
Err(EvaluationError::SnapshotMismatch)
);
let mut rewritten = approved.clone();
rewritten.approval.proposal_digest = classification_proposal_digest(&report);
assert_eq!(
evaluate_reviewed_golden_set(&report, &rewritten, |set| set == &approved),
Err(EvaluationError::UnverifiedApproval)
);
}
}
2 changes: 1 addition & 1 deletion docs/PRD.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Preserve every observed source, including standalone files and notes outside the

Library reads must finish within a bounded observation window or fail visibly without returning a partial classification. Slowly arriving pages cannot keep a run open indefinitely, and missing time budget must not be handled by silently dropping papers.

Read a complete Zotero Local API observation with one consistent library version and propose exactly one research disposition for every top-level bibliographic item. This consistency check does not establish an atomic provider snapshot. A record claiming a revision newer than the observed library invalidates the complete read; it must not be omitted or assigned a different revision to make the read pass. Each proposal retains the item key/version, exact matched metadata values, rule revision, linked child records, and any model receipt. Weak evidence and evidence that matches multiple specific disposition families must abstain into steward review. Duplicate DOI/title identities are review candidates only: intake never merges, deletes, or silently mutates Zotero records.
Read a complete Zotero Local API observation with one consistent library version and propose exactly one research disposition for every top-level bibliographic item. This consistency check does not establish an atomic provider snapshot. A record claiming a revision newer than the observed library invalidates the complete read; it must not be omitted or assigned a different revision to make the read pass. Each proposal retains the item key/version, exact matched metadata values, rule revision, linked child records, and any model receipt. Weak evidence and evidence that matches multiple specific disposition families must abstain into steward review. A local abstention retains its nonempty abstract exactly once, as matched evidence when applicable or otherwise as review context; decided items omit the review-only copy. Duplicate DOI/title identities are review candidates only: intake never merges, deletes, or silently mutates Zotero records.

For every connected duplicate component, accept externally verified steward decisions selecting one component-level canonical item. Produce a local-only manifest that binds decisions to the raw snapshot, complete item revisions, exact duplicate membership, current proposals and retained source metadata. Reject missing or inconsistent source inventory and invalid decisions before requesting approval. Changed retained evidence requires fresh independent approval, even when duplicate members are unchanged. Record every component source revision plus before, after, and rollback canonical-key mappings. Classification preserves every Zotero source record.

Expand Down
4 changes: 3 additions & 1 deletion docs/TRD.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ One monotonic five-minute budget covers page admission and complete-report accep

After count/byte validation and before accumulating each metadata page, every returned object's revision must be less than or equal to that page's library revision. This includes attachments, notes and annotations, not only bibliographic records. A higher revision returns the existing snapshot-consistency error immediately, with no next-page request or partial report. Zero, lower and equal revisions remain valid and are preserved exactly, including the unsigned maximum. Compare only within this metadata read of one local instance: Zotero 9's synced revisions and Zotero 10's local revisions are not interchangeable, and this condition is not a full-text endpoint contract or proof of atomicity. See the [revision admission evidence](doctoring/zotero_item_revision.md).

Every top-level bibliographic record receives exactly one proposed disposition. `NeedsStewardReview` also records a deterministic abstention reason so missing classification metadata, vocabulary unsupported by the current deterministic rules, and present-but-unmatched metadata are distinguishable. DOI duplicate identity normalization treats bare DOI values, `doi:`, `doi.org`, and legacy `dx.doi.org` resolver forms as the same identity when their normalized DOI is equal.
Every top-level bibliographic record receives exactly one proposed disposition. `NeedsStewardReview` also records a deterministic abstention reason so missing classification metadata, vocabulary unsupported by the current deterministic rules, and present-but-unmatched metadata are distinguishable. Its nonempty abstract is retained exactly once in the local report: an abstract that triggered conflicting rules remains in matched evidence, while other abstention abstracts use the review-only field. Non-abstained items omit that field. DOI duplicate identity normalization treats bare DOI values, `doi:`, `doi.org`, and legacy `dx.doi.org` resolver forms as the same identity when their normalized DOI is equal.

Duplicate decisions are independent of subject labels but share the complete source-evidence admission boundary. A reviewed set must match the raw-snapshot digest, item revisions, exact candidate membership and required v2 `proposal_digest`. Receipt comparison retains `SnapshotMismatch` precedence; shared structural/audit admission and all component/decision checks then run before the external verifier. The verifier authenticates the entire independently issued set, not a locally recomputed digest. Missing scope bindings fail deserialization and blank bindings fail admission; no legacy default or automatic reapproval exists. The manifest retains the verified proposal digest. Every operation records all component item revisions, before/after identity maps and exact rollback. Zotero records remain unchanged.

Expand All @@ -107,6 +107,8 @@ Structural, source, proposal, and label checks precede the external verifier. Bl
Provider deserialization captures each complete JSON object before projecting metadata. Snapshot hashing serializes the domain marker `conceptweave-zotero-snapshot-v2` followed by key-ordered pairs of that canonical source JSON and the actual typed classifier input. Unknown nested fields, array order, and omitted-versus-explicit default fields remain bound; changing a typed input after decoding also changes the digest. Synthetic offline typed items have no captured provider object and bind an explicit absent-source value alongside their typed input. Earlier reduced-content digests remain historical evidence and cannot establish this complete-content contract; regenerate the report and review artifacts and obtain fresh approval before any release or approved write.
A successful classification report carries an `audit_summary` whose snapshot, bibliographic, proposed-disposition, provenance-complete, abstention, duplicate-candidate, failure, and per-disposition counts are derived from the same in-memory immutable snapshot. Zotero item version zero remains a valid observed coordinate for never-synced Zotero 9 records; provenance completeness rejects a missing item key rather than inventing a positive-only version invariant. Reader failures return an error instead of a partial report; therefore a returned report records `failure_count=0` rather than hiding partial failures.

The local report can contain titles, tags, matched metadata, and abstention abstracts. It is sensitive steward-review material, remains outside the repository, and is not a publication artifact.

The report is local JSON and contains proposals rather than governance decisions. CLI output is restricted to a new direct child of canonical `/tmp` or the operating system temporary directory; relative paths, nested paths, existing paths, and symlinks are rejected, and create-new file semantics prevent overwrite/path-swap writes. Reviewed collection/tag changes can produce a pure local plan whose default mode is dry-run. The plan requires exact report and item preconditions, complete before/after/rollback arrays, externally verified authority, and preserved Zotero tag types; its fields are externally read-only after validation. Zotero 9 execute mode fails closed. Every receipt copies the plan's review, authority, server, Zotero version, library, rule, snapshot and proposal coordinates; dry-run reports every operation as not attempted and makes no Local API call. Execute mode preflights every item before the first write, advances the library precondition only from a directly verified write response, stops on the first adapter or response failure, and re-reads that item through the same boundary as observation only. Failed writes remain indeterminate regardless of observed metadata; no inverse is issued for them. Prior directly verified operations retain their inverse coordinates. The API key remains adapter-owned and absent from serializable structures.

PR #20 retains its rollback core and adapter: mixed-server rejection precedes reads; complete current-state checks precede inverse writes; only directly verified responses advance the library version. Every failed or invalid inverse response now remains indeterminate, retaining the complete operation, exact submitted request (including its library precondition), and optional complete readback. Matching restored or unchanged metadata does not prove causal completion or termination, and the failed inverse is absent from remaining work. Earlier directly verified restorations remain recorded; remaining operations are untouched only, not automatic retry authority. The operation-slice API still lacks original-write scope and independent authority; authoritative consumer adoption remains an open gate, including empty-slice and delayed-reconciliation handling.
Expand Down
Loading