diff --git a/README.md b/README.md index 50838ac..c881865 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,14 @@ If file-permission setup fails, the command stops before writing report content. An empty file may remain; inspect it before removing it. The command does not delete a pathname that another process may have replaced. +Create a small deterministic view of the next pending records for human review: + +```sh +cargo +1.98.0 run --bin conceptweave-zotero -- --review-batch /tmp/report.json /tmp/current-worksheet.json 25 /tmp/review-batch.json +``` + +The batch repeats on unchanged input and is not a reservation or assignment. It contains sensitive bibliographic context, must remain owner-only, and becomes a decision patch only after a steward fills every `reviewed_disposition`. + [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/ContextualWisdomLab/ConceptWeave) **Automatic, evidence-bound ontology and semantic-layer engineering for governed enterprise meaning.** diff --git a/crates/conceptweave-zotero/src/lib.rs b/crates/conceptweave-zotero/src/lib.rs index fcdc0ee..7930681 100644 --- a/crates/conceptweave-zotero/src/lib.rs +++ b/crates/conceptweave-zotero/src/lib.rs @@ -145,7 +145,7 @@ pub enum AbstentionReason { } /// Evidence for a deterministic proposed disposition. -#[derive(Debug, Deserialize, Serialize)] +#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize)] pub struct ClassificationEvidence { /// Metadata fields whose values matched. pub fields: Vec, @@ -156,7 +156,7 @@ pub struct ClassificationEvidence { } /// A single top-level bibliographic classification proposal. -#[derive(Debug, Deserialize, Serialize)] +#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize)] pub struct ClassifiedItem { /// Stable Zotero item key. pub item_key: String, @@ -1189,16 +1189,73 @@ pub struct StewardDecisionPatch { pub decisions: Vec, } +/// Maximum number of sensitive records exposed in one local steward review batch. +pub const MAX_REVIEW_BATCH_ITEMS: usize = 100; + +/// One pending decision with the report context required for human review. +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +pub struct StewardReviewBatchDecision { + /// Stable Zotero item key used to apply the completed decision. + pub item_key: String, + /// Exact item revision reviewed by the steward. + pub item_version: u64, + /// Zotero item type retained for local review context. + pub item_type: String, + /// Human-readable title retained in this owner-only artifact. + pub title: String, + /// Minimal abstract context retained only for abstained proposals. + pub review_abstract_note: Option, + /// Collection keys observed with the item. + pub collection_keys: Vec, + /// Typed tags observed with the item. + pub tags: Vec, + /// Deterministic proposal shown for comparison, never as approval. + pub proposed_disposition: Disposition, + /// Deterministic abstention reason when the proposal requires review. + pub abstention_reason: Option, + /// Deterministic evidence retained from the exact report snapshot. + pub evidence: ClassificationEvidence, + /// Blank slot for the human steward to fill with non-abstention truth. + pub reviewed_disposition: Option, +} + +/// Deterministic owner-only view of the next pending steward decisions. +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +pub struct StewardReviewBatch { + /// Zotero library revision shared with the report and worksheet. + pub library_version: u64, + /// Classifier revision whose proposals are being reviewed. + pub rule_revision: String, + /// Canonical digest of the complete raw snapshot. + pub snapshot_digest: String, + /// Opaque identity retained unchanged when completed decisions become a patch. + pub proposal_digest: String, + /// Unresolved source records, separate from blank bibliographic decision slots. + pub pending_source_count: usize, + /// Pending decisions before this non-reserving view was created. + pub remaining_count: usize, + /// First pending decisions in canonical item-key order. + pub decisions: Vec, +} + /// A classification report cannot safely produce a review worksheet. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum WorksheetError { /// Report identity or coverage is incomplete, duplicated, or inconsistent. InvalidReport, + /// Requested review batch size is outside the supported local bound. + InvalidBatchLimit, + /// The canonical worksheet contains no blank decision. + NoPendingDecisions, } impl fmt::Display for WorksheetError { fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { - formatter.write_str("classification report is invalid for steward review") + formatter.write_str(match self { + Self::InvalidReport => "classification report is invalid for steward review", + Self::InvalidBatchLimit => "review batch limit must be between 1 and 100", + Self::NoPendingDecisions => "steward worksheet has no pending decisions", + }) } } @@ -1220,6 +1277,17 @@ pub fn build_steward_review_worksheet( for item in &report.classified_items { if (item.proposed_disposition == Disposition::NeedsStewardReview) != item.abstention_reason.is_some() + || (item.proposed_disposition != Disposition::NeedsStewardReview + && item.review_abstract_note.is_some()) + || item + .review_abstract_note + .as_ref() + .is_some_and(|review_abstract| { + item.evidence + .field_values + .values() + .any(|value| value == review_abstract) + }) { return Err(WorksheetError::InvalidReport); } @@ -1279,6 +1347,57 @@ pub fn assess_steward_review_progress( }) } +/// Builds a deterministic, non-reserving view of the next pending review decisions. +pub fn build_steward_review_batch( + report: &ClassificationReport, + worksheet: &StewardReviewWorksheet, + limit: usize, +) -> Result { + if !(1..=MAX_REVIEW_BATCH_ITEMS).contains(&limit) { + return Err(WorksheetError::InvalidBatchLimit); + } + let progress = assess_steward_review_progress(report, worksheet)?; + if progress.remaining_count == 0 { + return Err(WorksheetError::NoPendingDecisions); + } + let classified_by_key: BTreeMap<_, _> = report + .classified_items + .iter() + .map(|item| (item.item_key.as_str(), item)) + .collect(); + let decisions = worksheet + .decisions + .iter() + .filter(|decision| decision.reviewed_disposition.is_none()) + .take(limit) + .map(|decision| { + let item = classified_by_key[decision.item_key.as_str()]; + StewardReviewBatchDecision { + item_key: item.item_key.clone(), + item_version: item.item_version, + item_type: item.item_type.clone(), + title: item.title.clone(), + review_abstract_note: item.review_abstract_note.clone(), + collection_keys: item.collection_keys.clone(), + tags: item.tags.clone(), + proposed_disposition: item.proposed_disposition, + abstention_reason: item.abstention_reason, + evidence: item.evidence.clone(), + reviewed_disposition: None, + } + }) + .collect(); + Ok(StewardReviewBatch { + library_version: report.library_version, + rule_revision: report.rule_revision.clone(), + snapshot_digest: report.snapshot_digest.clone(), + proposal_digest: progress.proposal_digest, + pending_source_count: progress.pending_source_count, + remaining_count: progress.remaining_count, + decisions, + }) +} + /// Applies one snapshot-bound decision patch without overwriting conflicting review work. pub fn apply_steward_decision_patch( report: &ClassificationReport, diff --git a/crates/conceptweave-zotero/src/main.rs b/crates/conceptweave-zotero/src/main.rs index 390f9d3..779a69a 100644 --- a/crates/conceptweave-zotero/src/main.rs +++ b/crates/conceptweave-zotero/src/main.rs @@ -2,9 +2,10 @@ #![cfg_attr(coverage_nightly, feature(coverage_attribute))] use conceptweave_zotero::{ - ClassificationReport, GoldenSetApproval, StewardDecisionPatch, StewardReviewWorksheet, - apply_steward_decision_patch, assess_steward_review_progress, build_steward_review_worksheet, - read_local_snapshot, reviewed_golden_set_from_worksheet, + ClassificationReport, GoldenSetApproval, MAX_REVIEW_BATCH_ITEMS, StewardDecisionPatch, + StewardReviewWorksheet, apply_steward_decision_patch, assess_steward_review_progress, + build_steward_review_batch, build_steward_review_worksheet, read_local_snapshot, + reviewed_golden_set_from_worksheet, }; use serde::de::DeserializeOwned; use std::collections::BTreeSet; @@ -13,7 +14,7 @@ use std::fs::{self, File, OpenOptions}; use std::io::{self, BufWriter, Read, Write}; use std::path::{Path, PathBuf}; -const USAGE: &str = "usage: conceptweave-zotero /tmp/REPORT.json | --worksheet /tmp/REPORT.json /tmp/WORKSHEET.json | --review-progress /tmp/REPORT.json /tmp/WORKSHEET.json /tmp/PROGRESS.json | --apply-decision-patch /tmp/REPORT.json /tmp/CURRENT_WORKSHEET.json /tmp/PATCH.json /tmp/UPDATED_WORKSHEET.json | --finalize /tmp/REPORT.json /tmp/WORKSHEET.json /tmp/APPROVAL.json /tmp/GOLDEN.json"; +const USAGE: &str = "usage: conceptweave-zotero /tmp/REPORT.json | --worksheet /tmp/REPORT.json /tmp/WORKSHEET.json | --review-progress /tmp/REPORT.json /tmp/WORKSHEET.json /tmp/PROGRESS.json | --review-batch /tmp/REPORT.json /tmp/CURRENT_WORKSHEET.json LIMIT /tmp/BATCH.json | --apply-decision-patch /tmp/REPORT.json /tmp/CURRENT_WORKSHEET.json /tmp/PATCH.json /tmp/UPDATED_WORKSHEET.json | --finalize /tmp/REPORT.json /tmp/WORKSHEET.json /tmp/APPROVAL.json /tmp/GOLDEN.json"; const MAX_ARTIFACT_BYTES: u64 = 16 * 1024 * 1024; #[derive(Debug, PartialEq, Eq)] @@ -28,6 +29,12 @@ enum OutputRequest { worksheet: String, output: String, }, + ReviewBatch { + report: String, + worksheet: String, + limit: usize, + output: String, + }, ApplyDecisionPatch { report: String, worksheet: String, @@ -79,6 +86,37 @@ where worksheet, output, } + } else if first == "--review-batch" { + let report = args + .next() + .ok_or("--review-batch requires report, worksheet, limit, and output")?; + let worksheet = args + .next() + .ok_or("--review-batch requires report, worksheet, limit, and output")?; + let limit = args + .next() + .ok_or("--review-batch requires report, worksheet, limit, and output")?; + let output = args + .next() + .ok_or("--review-batch requires report, worksheet, limit, and output")?; + if BTreeSet::from([report.as_str(), worksheet.as_str(), output.as_str()]).len() != 3 { + return Err("review batch artifact paths must differ"); + } + if limit.is_empty() || !limit.bytes().all(|byte| byte.is_ascii_digit()) { + return Err("review batch limit must be an unsigned decimal integer"); + } + let limit = limit + .parse::() + .map_err(|_| "review batch limit is out of range")?; + if !(1..=MAX_REVIEW_BATCH_ITEMS).contains(&limit) { + return Err("review batch limit must be between 1 and 100"); + } + OutputRequest::ReviewBatch { + report, + worksheet, + limit, + output, + } } else if first == "--apply-decision-patch" { let report = args .next() @@ -447,6 +485,28 @@ fn main() -> Result<(), Box> { let progress = assess_steward_review_progress(&report, &worksheet)?; write_private_output(&output, &serde_json::to_vec_pretty(&progress)?)?; } + OutputRequest::ReviewBatch { + report, + worksheet, + limit, + output, + } => { + let output = validate_output_path(&output)?; + let (report, report_identity): (ClassificationReport, _) = + read_private_json(&report).map_err(|error| label_input("report", error))?; + let (worksheet, worksheet_identity): (StewardReviewWorksheet, _) = + read_private_json(&worksheet).map_err(|error| label_input("worksheet", error))?; + if report_identity == worksheet_identity { + return Err(io::Error::new( + io::ErrorKind::InvalidInput, + "review batch inputs must be distinct files", + ) + .into()); + } + let batch = build_steward_review_batch(&report, &worksheet, limit)?; + let content = serde_json::to_vec_pretty(&batch)?; + write_private_output(&output, &content)?; + } OutputRequest::ApplyDecisionPatch { report, worksheet, @@ -717,6 +777,58 @@ mod tests { ); } + #[test] + fn review_batch_mode_requires_distinct_paths_and_decimal_limit() { + let report = "/tmp/report.json"; + let worksheet = "/tmp/worksheet.json"; + let output = "/tmp/batch.json"; + assert_eq!( + parse_output_request(vec!["--review-batch", report, worksheet, "25", output]), + Ok(OutputRequest::ReviewBatch { + report: report.to_owned(), + worksheet: worksheet.to_owned(), + limit: 25, + output: output.to_owned(), + }) + ); + for limit in [ + "", + "0", + "101", + " 1", + "+1", + "-1", + "one", + "9999999999999999999999999999999999999999", + ] { + assert!( + parse_output_request(vec!["--review-batch", report, worksheet, limit, output]) + .is_err() + ); + } + assert!(parse_output_request(vec!["--review-batch"]).is_err()); + assert!(parse_output_request(vec!["--review-batch", report]).is_err()); + assert!(parse_output_request(vec!["--review-batch", report, worksheet]).is_err()); + assert!(parse_output_request(vec!["--review-batch", report, worksheet, "25"]).is_err()); + assert!( + parse_output_request(vec!["--review-batch", report, report, "25", output]).is_err() + ); + assert!( + parse_output_request(vec!["--review-batch", report, worksheet, "25", report]).is_err() + ); + assert!( + parse_output_request(vec![ + "--review-batch", + report, + worksheet, + "25", + output, + "extra", + ]) + .is_err() + ); + } + #[cfg(unix)] #[test] fn private_json_input_is_owner_only_regular_bounded_and_valid() { diff --git a/crates/conceptweave-zotero/tests/finalization_artifact_identity.rs b/crates/conceptweave-zotero/tests/finalization_artifact_identity.rs index c1cda8b..8d74789 100644 --- a/crates/conceptweave-zotero/tests/finalization_artifact_identity.rs +++ b/crates/conceptweave-zotero/tests/finalization_artifact_identity.rs @@ -110,6 +110,16 @@ fn artifact_commands_reject_distinct_path_spellings_for_one_input_file() { ]) .status() .unwrap(); + let batch_status = Command::new(env!("CARGO_BIN_EXE_conceptweave-zotero")) + .args([ + "--review-batch", + &report_path, + &worksheet_path, + "1", + output.to_str().unwrap(), + ]) + .status() + .unwrap(); let _ = fs::remove_file(&input); let _ = fs::remove_file(&output); @@ -125,4 +135,8 @@ fn artifact_commands_reject_distinct_path_spellings_for_one_input_file() { !patch_status.success(), "decision patching must reject path spellings that resolve to one input artifact" ); + assert!( + !batch_status.success(), + "review batching must reject path spellings that resolve to one input artifact" + ); } diff --git a/crates/conceptweave-zotero/tests/steward_review_batch.rs b/crates/conceptweave-zotero/tests/steward_review_batch.rs new file mode 100644 index 0000000..d22e840 --- /dev/null +++ b/crates/conceptweave-zotero/tests/steward_review_batch.rs @@ -0,0 +1,170 @@ +use conceptweave_zotero::{ + Disposition, ItemData, StewardDecisionPatch, WorksheetError, ZoteroItem, + apply_steward_decision_patch, build_steward_review_batch, build_steward_review_worksheet, + classify_snapshot, +}; + +fn item(key: &str, title: &str, abstract_note: &str) -> ZoteroItem { + ZoteroItem { + source_record: None, + key: key.into(), + version: 7, + data: ItemData { + item_type: "journalArticle".into(), + title: title.into(), + abstract_note: abstract_note.into(), + doi: String::new(), + parent_item: String::new(), + collections: vec!["COLLECTION".into()], + tags: vec![], + }, + } +} + +#[test] +fn review_batch_is_deterministic_bounded_and_patch_compatible() { + let report = classify_snapshot( + "9.0.6".into(), + None, + 42, + vec![ + item("C", "unmatched C", "context C"), + item("A", "ontology alignment", ""), + item("B", "unmatched B", "context B"), + ], + ); + let mut worksheet = build_steward_review_worksheet(&report).unwrap(); + worksheet.decisions[0].reviewed_disposition = Some(Disposition::AlignmentVersioning); + + let batch = build_steward_review_batch(&report, &worksheet, 1).unwrap(); + assert_eq!(batch.remaining_count, 2); + assert_eq!(batch.decisions.len(), 1); + assert_eq!(batch.decisions[0].item_key, "B"); + assert_eq!(batch.decisions[0].title, "unmatched B"); + assert_eq!( + batch.decisions[0].review_abstract_note.as_deref(), + Some("context B") + ); + assert_eq!(batch.decisions[0].reviewed_disposition, None); + assert_eq!( + build_steward_review_batch(&report, &worksheet, 1).unwrap(), + batch + ); + let serialized = serde_json::to_string(&batch).unwrap(); + assert_eq!(serialized.matches("context B").count(), 1); + for omitted in [ + "snapshot_items", + "child_item_keys", + "model_receipt", + "audit_summary", + ] { + assert!(!serialized.contains(omitted)); + } + + let mut completed_batch = batch.clone(); + completed_batch.decisions[0].reviewed_disposition = Some(Disposition::OutOfScope); + let patch: StewardDecisionPatch = + serde_json::from_value(serde_json::to_value(&completed_batch).unwrap()).unwrap(); + let updated = apply_steward_decision_patch(&report, &worksheet, &patch).unwrap(); + assert_eq!( + updated.decisions[1].reviewed_disposition, + Some(Disposition::OutOfScope) + ); + + assert_eq!( + build_steward_review_batch(&report, &worksheet, 0), + Err(WorksheetError::InvalidBatchLimit) + ); + assert_eq!( + build_steward_review_batch(&report, &worksheet, 101), + Err(WorksheetError::InvalidBatchLimit) + ); +} + +#[test] +fn review_batch_rejects_invalid_or_complete_workloads() { + let report = classify_snapshot( + "9.0.6".into(), + None, + 42, + vec![item("A", "unmatched", "review context")], + ); + let mut worksheet = build_steward_review_worksheet(&report).unwrap(); + + let mut invalid = worksheet.clone(); + invalid.decisions[0].item_version += 1; + assert_eq!( + build_steward_review_batch(&report, &invalid, 1), + Err(WorksheetError::InvalidReport) + ); + + worksheet.decisions[0].reviewed_disposition = Some(Disposition::OutOfScope); + assert_eq!( + build_steward_review_batch(&report, &worksheet, 1), + Err(WorksheetError::NoPendingDecisions) + ); + assert_eq!( + WorksheetError::InvalidBatchLimit.to_string(), + "review batch limit must be between 1 and 100" + ); + assert_eq!( + WorksheetError::NoPendingDecisions.to_string(), + "steward worksheet has no pending decisions" + ); + + let mut duplicated_abstract_report = classify_snapshot( + "9.0.6".into(), + None, + 42, + vec![item("A", "unmatched", "review context")], + ); + duplicated_abstract_report.classified_items[0] + .evidence + .field_values + .insert("abstractNote".into(), "review context".into()); + assert_eq!( + build_steward_review_worksheet(&duplicated_abstract_report), + Err(WorksheetError::InvalidReport) + ); + + let mut decided_abstract_report = classify_snapshot( + "9.0.6".into(), + None, + 42, + vec![item("A", "ontology alignment", "")], + ); + decided_abstract_report.classified_items[0].review_abstract_note = Some("unexpected".into()); + assert_eq!( + build_steward_review_worksheet(&decided_abstract_report), + Err(WorksheetError::InvalidReport) + ); +} + +#[test] +fn batch_keeps_pending_scope_separate_from_bibliographic_slots() { + let mut source = item("PRIVATE_SOURCE", "unresolved source", ""); + source.data.item_type = "attachment".into(); + let report = classify_snapshot( + "9.0.6".into(), + None, + 42, + vec![item("A", "unmatched", "context"), source], + ); + let mut worksheet = build_steward_review_worksheet(&report).unwrap(); + let batch = build_steward_review_batch(&report, &worksheet, 1).unwrap(); + let value = serde_json::to_value(&batch).unwrap(); + assert_eq!(value["pending_source_count"], 1); + assert_eq!(value["proposal_digest"], worksheet.proposal_digest); + assert_eq!(batch.remaining_count, 1); + assert!(!value.to_string().contains("PRIVATE_SOURCE")); + worksheet.decisions[0].reviewed_disposition = Some(Disposition::OutOfScope); + assert_eq!( + build_steward_review_batch(&report, &worksheet, 1), + Err(WorksheetError::NoPendingDecisions) + ); + assert!( + !conceptweave_zotero::assess_steward_review_progress(&report, &worksheet) + .unwrap() + .complete + ); +} diff --git a/crates/conceptweave-zotero/tests/steward_review_batch_cli.rs b/crates/conceptweave-zotero/tests/steward_review_batch_cli.rs new file mode 100644 index 0000000..9e25b5e --- /dev/null +++ b/crates/conceptweave-zotero/tests/steward_review_batch_cli.rs @@ -0,0 +1,118 @@ +#![cfg(unix)] + +use conceptweave_zotero::{ + Disposition, ItemData, ZoteroItem, build_steward_review_worksheet, classify_snapshot, +}; +use std::fs; +use std::os::unix::fs::PermissionsExt; +use std::path::Path; +use std::process::Command; + +#[test] +fn review_batch_cli_writes_sensitive_context_owner_only() { + let report = classify_snapshot( + "9.0.6".into(), + None, + 42, + vec![item("B", "unmatched B"), item("A", "unmatched A")], + ); + let worksheet = build_steward_review_worksheet(&report).unwrap(); + let report_path = private_input("batch-cli-report", &report); + let worksheet_path = private_input("batch-cli-worksheet", &worksheet); + let output_path = temp_path("batch-cli-output"); + let _ = fs::remove_file(&output_path); + + assert!(run_batch(&report_path, &worksheet_path, "1", &output_path).success()); + let output: serde_json::Value = + serde_json::from_slice(&fs::read(&output_path).unwrap()).unwrap(); + assert_eq!(output["remaining_count"], 2); + assert_eq!(output["decisions"].as_array().unwrap().len(), 1); + assert_eq!(output["decisions"][0]["item_key"], "A"); + assert_eq!( + output["decisions"][0]["reviewed_disposition"], + serde_json::Value::Null + ); + assert!(output.get("snapshot_items").is_none()); + assert!(output.get("duplicate_candidates").is_none()); + assert_eq!( + fs::metadata(&output_path).unwrap().permissions().mode() & 0o777, + 0o600 + ); + + for path in [report_path, worksheet_path, output_path] { + fs::remove_file(path).unwrap(); + } +} + +#[test] +fn review_batch_cli_emits_nothing_for_complete_or_existing_output() { + let report = classify_snapshot("9.0.6".into(), None, 42, vec![item("A", "unmatched")]); + let mut worksheet = build_steward_review_worksheet(&report).unwrap(); + worksheet.decisions[0].reviewed_disposition = Some(Disposition::OutOfScope); + let report_path = private_input("batch-complete-report", &report); + let worksheet_path = private_input("batch-complete-worksheet", &worksheet); + let absent_output = temp_path("batch-complete-output"); + let existing_output = temp_path("batch-existing-output"); + let _ = fs::remove_file(&absent_output); + let _ = fs::remove_file(&existing_output); + + assert!(!run_batch(&report_path, &worksheet_path, "1", &absent_output).success()); + assert!(!absent_output.exists()); + fs::write(&existing_output, b"preserve me").unwrap(); + assert!(!run_batch(&report_path, &worksheet_path, "1", &existing_output).success()); + assert_eq!(fs::read(&existing_output).unwrap(), b"preserve me"); + + for path in [report_path, worksheet_path, existing_output] { + fs::remove_file(path).unwrap(); + } +} + +fn item(key: &str, title: &str) -> ZoteroItem { + ZoteroItem { + source_record: None, + key: key.into(), + version: 7, + data: ItemData { + item_type: "journalArticle".into(), + title: title.into(), + abstract_note: "review context".into(), + doi: String::new(), + parent_item: String::new(), + collections: vec![], + tags: vec![], + }, + } +} + +fn private_input(name: &str, value: &impl serde::Serialize) -> std::path::PathBuf { + let path = temp_path(name); + let _ = fs::remove_file(&path); + fs::write(&path, serde_json::to_vec(value).unwrap()).unwrap(); + fs::set_permissions(&path, fs::Permissions::from_mode(0o600)).unwrap(); + path +} + +fn run_batch( + report: &Path, + worksheet: &Path, + limit: &str, + output: &Path, +) -> std::process::ExitStatus { + Command::new(env!("CARGO_BIN_EXE_conceptweave-zotero")) + .args([ + "--review-batch", + report.to_str().unwrap(), + worksheet.to_str().unwrap(), + limit, + output.to_str().unwrap(), + ]) + .status() + .unwrap() +} + +fn temp_path(name: &str) -> std::path::PathBuf { + std::env::temp_dir().join(format!( + "conceptweave-zotero-{}-{name}.json", + std::process::id() + )) +} diff --git a/crates/conceptweave-zotero/tests/steward_review_finalization.rs b/crates/conceptweave-zotero/tests/steward_review_finalization.rs index 46c0aa0..9f014fe 100644 --- a/crates/conceptweave-zotero/tests/steward_review_finalization.rs +++ b/crates/conceptweave-zotero/tests/steward_review_finalization.rs @@ -358,7 +358,7 @@ fn finalization_rejects_stale_worksheet_even_with_current_approval_coordinates() match changed_field { 0 => report.classified_items[0].title.push_str(" changed"), 1 => report.classified_items[0].evidence.field_values.clear(), - _ => report.classified_items[0].review_abstract_note = Some("changed context".into()), + _ => report.classified_items[1].review_abstract_note = Some("changed context".into()), } let current_receipt = approval(&report, &worksheet); assert_eq!( diff --git a/docs/PRD.md b/docs/PRD.md index 5f47731..05299a0 100644 --- a/docs/PRD.md +++ b/docs/PRD.md @@ -94,6 +94,7 @@ The worksheet's own required content identity must match the current report inde Operators must be able to accumulate small steward-reviewed decision sets without hand-merging the complete worksheet. Each patch binds the original library version, classifier revision, snapshot and proposal/retained-content digests, item key and item revision. Regenerating a worksheet after content changes cannot make an older patch valid. Missing content binding requires a new review-bound patch, never automatic backfill. Empty, duplicate, unknown, stale or abstention decisions fail atomically. Identical replay is idempotent; conflicting decisions cannot overwrite review work. Applying a patch does not confer independent approval, full-text review provenance or publication authority. The offline CLI must read the saved report, current worksheet, and decision patch as three distinct owner-only file identities and create a separate updated worksheet. It must never overwrite the current worksheet, reread Zotero, or emit output after invalid input. +Operators must be able to extract up to 100 blank bibliographic decisions with the exact context needed for review. Batches preserve current content identity and separately show unresolved source count; no blank paper decisions does not mean all sources are resolved. Ordering is deterministic, decided rows are skipped and unchanged inputs reproduce the same batch. Creation is neither assignment nor progress; only an accepted completed patch increases unverified decision coverage, never independent approval or applied reclassification. Evaluate classifier quality only against a steward-reviewed local golden set whose governance receipt is externally verified and binds both the complete source/classifier-input snapshot and every current proposal field, in addition to the item-key/item-version coordinates. Same-version changes to unmodeled provider metadata, absent/default fields, classifier inputs, predictions or supporting evidence must invalidate the corresponding binding. Evaluation recomputes proposal identity before contacting governance; a locally changed digest cannot renew an approval. Legacy unbound approvals require reissuance, never automatic backfill. Abstention is a prediction outcome, never an approved truth label. Evaluation emits the verified library revision, rule revision, opaque snapshot and proposal digests, and aggregate counts for exact matches, abstentions, and per-disposition true-positive/predicted/expected totals; it must not copy Zotero keys, reviewer identity, or bibliographic text into the result. diff --git a/docs/TRD.md b/docs/TRD.md index 0ac6f39..e2100de 100644 --- a/docs/TRD.md +++ b/docs/TRD.md @@ -123,6 +123,8 @@ Offline input admission pins the checked canonical parent plus file name for met `apply_steward_decision_patch` rebuilds the canonical worksheet and validates the complete current worksheet, including rejection of pre-existing reviewed abstention. It validates a nonempty patch against library version, rule revision, snapshot digest and its own required `proposal_digest`; a fresh worksheet cannot renew an older patch's reviewed content. Missing binding fails deserialization; blank or stale binding fails before updates, without backfill. Every update names a unique canonical item key and exact revision with a non-abstention disposition. Updates apply to a clone, so duplicate, unknown, stale or conflicting decisions reject the whole batch without partial state. Identical replay remains idempotent. This is the owner boundary for a later private CLI, not independent approval or full-text/write authority. `conceptweave-zotero --apply-decision-patch /tmp/REPORT.json /tmp/CURRENT_WORKSHEET.json /tmp/PATCH.json /tmp/UPDATED_WORKSHEET.json` exposes that contract through the existing private artifact boundary. Four textual paths and all three opened input device/inode identities must differ. Inputs are bounded, regular, single-link, exact `0600`, opened through the checked canonical parent with no-follow/nonblocking flags. The required patch proposal binding is deserialized and passed unchanged to the owner validator, never synthesized by the CLI. Validation failure leaves no output; existing output is preserved even for a valid patch. Serialization precedes create-new output, while a later write failure may retain private partial files without cleanup or retry. Identical replay may create another equal worksheet. No Zotero read or authority verifier runs. +`conceptweave-zotero --review-batch /tmp/REPORT.json /tmp/CURRENT_WORKSHEET.json LIMIT /tmp/BATCH.json` validates distinct private inputs and emits the first 1–100 blank bibliographic decisions in canonical order. It skips decided rows and rejects invalid worksheets, reviewed abstention or no remaining slots before output. Each sensitive row retains exact key/version, type, title, minimized abstract, collections, tags, proposal/reason/evidence and a blank decision. The batch carries snapshot coordinates plus the already validated `proposal_digest` and `pending_source_count`; remaining slots and unresolved sources are separate counts. No slots left does not prove source completion. Snapshot-item arrays, child keys, model receipts, audit/duplicate aggregates and approval identity remain omitted. Filled metadata batches retain their required digest when deserialized as `StewardDecisionPatch`; extra context is not authority and no full-text review provenance is granted. Original report and current worksheet are revalidated at application. Repeated input gives an equal batch without reservation, assignment or exclusivity. + The local report can contain titles, tags, matched metadata, and abstention abstracts. It is sensitive steward-review material, remains outside the repository, and is not a publication artifact. The report is local JSON and contains proposals rather than governance decisions. On supported Unix platforms, CLI output is restricted to a new owner-readable/writable (`0600`) direct child of canonical `/tmp` or the operating system temporary directory; exact permissions are restored after umask application, and other platforms fail closed. Relative paths, nested paths, existing paths, and symlinks are rejected, and create-new file semantics prevent overwrite/path-swap writes. Reviewed collection/tag changes can produce a pure local plan whose default mode is dry-run. The plan requires exact report and item preconditions, complete before/after/rollback arrays, externally verified authority, and preserved Zotero tag types; its fields are externally read-only after validation. Zotero 9 execute mode fails closed. Every receipt copies the plan's review, authority, server, Zotero version, library, rule, snapshot and proposal coordinates; dry-run reports every operation as not attempted and makes no Local API call. Execute mode preflights every item before the first write, advances the library precondition only from a directly verified write response, stops on the first adapter or response failure, and re-reads that item through the same boundary as observation only. Failed writes remain indeterminate regardless of observed metadata; no inverse is issued for them. Prior directly verified operations retain their inverse coordinates. The API key remains adapter-owned and absent from serializable structures. diff --git a/docs/adr/0006-zotero-research-intake.md b/docs/adr/0006-zotero-research-intake.md index a41a7f7..bb697f7 100644 --- a/docs/adr/0006-zotero-research-intake.md +++ b/docs/adr/0006-zotero-research-intake.md @@ -59,6 +59,8 @@ Incremental steward work is integrated by a snapshot-bound decision patch rather The offline CLI consumes the saved report, current worksheet, and patch as distinct owner-only file identities and emits only a separate create-new worksheet. Reusing the existing private artifact boundary keeps the command local, bounded, and fail-closed without introducing a second review service or repository. The CLI never rereads Zotero, overwrites the source worksheet, or verifies approval. +Human review uses deterministic batches of at most 100 pending records. A batch projects only the matching report context and blank decision slots, remains owner-only, and can become the existing patch wire shape after a steward fills each decision. We choose a repeatable view instead of reservation state because the current campaign has no independent assignment service or cross-product consumer; concurrency ownership must be added only when such a contract exists. Creating a batch does not advance review or approval KPIs. + The reader fails closed above 50,000 items or 256 MiB of cumulative response bodies, while retaining the 8 MiB per-page bound, finite request timeouts, redirect denial, total-count checks, snapshot-version checks, and duplicate-key detection. Pagination, consistency, resource-budget, and provider-contract behavior are separated from the narrow `ureq` transport so deterministic tests exercise the production reader core rather than excluding the entire reader from coverage. In the context of reading every bibliographic source before classification, facing individually timely pages that can cumulatively hold a run open for days, we decided for a five-minute monotonic admission/completion budget in the existing reader and against rejecting legitimate short pages or adding another transport, to bound accepted work without excluding papers, accepting that an already-started request or classification computation can finish after the limit before its result is rejected. This is an application read limit, not a model timeout, hard process-cancellation deadline, wall-clock/suspend guarantee or atomic snapshot claim. Each page is checked before fetch and after return, and the complete report is checked before return. The stdlib clock has a private deterministic test seam; public APIs, provider timeouts and data/byte ceilings are unchanged. The [deadline doctoring](../doctoring/zotero_metadata_deadline.md) records the original review, RED/GREEN, alternatives and exact verification. This amendment remains Proposed and grants no Zotero mutation authority. @@ -124,6 +126,14 @@ GREEN source `f6735b585022aac1c8ceff86c150d9b64fd77ec2` passed 68 tests across 1 ## Alternatives considered +### Proposed review-batch scope projection amendment — September 7 + +PR33 exports sensitive bounded metadata views. Normal merge `2f3962e` preserves its context minimization guards and PR32's shared source/patch/private-file contracts. The existing batch-to-patch test then failed because the view omitted the required proposal identity. RED `7373579` compiled with one passing and two failing tests for patch compatibility and omitted pending-source scope. `ce15389` copies the two already validated progress fields into the existing batch: opaque proposal identity and pending-source count. No new digest, record class, full-text capture or authority mechanism is introduced. + +We reject recomputing a separate batch hash because the completed patch must retain the exact reviewed report scope. We reject silently backfilling the digest at conversion because that could relabel an old view as current. We reject treating unresolved records as paper slots; the batch keeps both counts and no-pending-decisions means only no blank bibliographic slots. A test fills the last paper slot while a standalone source remains and verifies batch exhaustion alongside incomplete source progress. Existing repeated-view determinism, 1–100 limit, blank-start decisions and minimized abstract guards remain. The view is not a reservation or approval, and stripping metadata context does not grant full-text provenance. Legacy batch artifacts require regeneration; later batch consumers must preserve these fields before protected release. + +The first complete verification exposed an inherited test that added review-only abstract context to a non-abstained proposal. PR33's retained privacy rule correctly rejects that malformed report before stale-identity comparison. Test-only `acbaf1d` instead mutates the existing abstained item, preserving the intended structurally valid content-change test and its `SnapshotMismatch` expectation. No production guard or error ordering changed. Exact completed-view context validation remains the subsequent PR34 owner; metadata patch compatibility alone does not prove the displayed context was unchanged. + ### Proposed decision CLI integration amendment — September 7 PR32 exposes the patch owner as an offline command. Normal merge `b7cae29` retains the original CLI delta and PR31's required proposal binding with prior private-file protections. The existing arm reads three distinct private inputs, delegates the typed patch unchanged to the canonical validator, serializes the returned worksheet and creates a new output. We retain that implementation rather than add another validator or infer a patch digest in the CLI. Two synthetic fixture constructors adopt the required binding; no production source is copied or new dependency introduced. diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 897e99e..92cdc6c 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -6,6 +6,14 @@ This file records code-current product and technical gaps. Exact PR/check/run co ## September 6 source inventory checkpoint +### September 7 PR33 batch scope repair + +Original PR33 `d82b2f2896bc4cfef5d34ac6f6f83f7cee1072f6` passed 166 tests/36 suites. Normal merge `2f3962e` retains it and PR32 `ea25437c54b4e452f008f6b1ccad92d5516dd7b9`, preserving original abstract minimization guards with the shared validator. The existing batch-to-patch test failed for missing proposal identity. Explicit RED `7373579` compiled with one passing and two failing tests for patch compatibility and pending-source projection. `ce15389` passes existing validated progress identity and pending count into the batch, without new hashing or authority. No blank slots means exhausted paper decisions, not resolved source scope. Independent bounded review found no added defect. + +Full verification then exposed an inherited stale-review fixture assigning review-only context to a non-abstained proposal; the retained privacy gate correctly returned InvalidReview earlier. Test-only `acbaf1d` mutates the actual abstained item instead, retaining structurally valid changed-content rejection as SnapshotMismatch. Final source passes 216 tests/36 suites including three doctests, strict Clippy, warnings-denied rustdoc, formatting, CI contract and diff checks. Unchanged coverage passes 340/340 reported functions, 3,120/3,120 normalized regions and 568/568 normalized branches. Raw 4,204/4,266 lines, 6,425/6,532 regions and 523/568 branches are not 100%. Logs: `/tmp/conceptweave-pr33-{baseline,integrated,binding-red,verified,final,clippy-final,rustdoc-final,coverage-final}.log`; `verified` is the failed intermediate run, not final GREEN. + +PRD/TRD/Proposed ADR0006 distinguish sensitive batch views, exact metadata identity, pending counts and no reservation/approval. PR34 owns completed-view context validation; plain patch compatibility does not prove displayed content integrity or full-text review. Root/later consumers remain unadopted. Actual decisions and independent approvals remain 0/3,715 plus four unresolved standalone sources. Native Visual Inspection was retried but the Mac is locked; no new screenshot, Zotero write, hosted GREEN, protected merge or release is claimed. Keep Draft. + ### September 7 PR32 decision CLI continuity verification Original PR32 `1e711728d83efc1e60fc3d43ba0c67c467dd6a43` passed 161 tests/34 suites. Normal merge `b7cae29` retains it and PR31 `eaf248afd33dcac477daf1e3a31d79d47c5cbb69`; two synthetic patch constructors adopt required proposal identity, and integrated tests pass 210/34. The thin apply arm remains unchanged: private distinct inputs go to the canonical validator without binding backfill; output is serialized before create-new writing. No additional production defect or new RED is claimed.