Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
70ddedc
test(zotero): require explicit worksheet CLI mode
seonghobae Sep 4, 2026
86b5d81
feat(zotero): export steward worksheet from CLI
seonghobae Sep 4, 2026
571aeec
docs(zotero): document live worksheet export
seonghobae Sep 4, 2026
76abb73
fix(zotero): describe both local output modes
seonghobae Sep 4, 2026
efed098
style(zotero): format CLI parser
seonghobae Sep 4, 2026
4e2047e
test(zotero): require one-snapshot review artifact pair
seonghobae Sep 4, 2026
e75ee51
fix(zotero): bind and clean review artifact pair
seonghobae Sep 4, 2026
0356636
docs(zotero): require paired review outputs
seonghobae Sep 4, 2026
63d1913
style(zotero): format paired output parser
seonghobae Sep 4, 2026
44dc487
refactor(zotero): serialize review outputs before creation
seonghobae Sep 4, 2026
92d67ea
style(zotero): format private output helper
seonghobae Sep 4, 2026
ec94738
fix(zotero): propagate paired output errors
seonghobae Sep 4, 2026
382da03
test(zotero): cover private output creation failure
seonghobae Sep 4, 2026
dc60e6a
fix(zotero): document paired CLI usage
seonghobae Sep 4, 2026
2efa1ab
style(zotero): format CLI usage
seonghobae Sep 4, 2026
c6eb0cd
Merge current worksheet parent into paired artifact CLI
seonghobae Sep 4, 2026
8fe7ba4
Merge repaired write receipt evidence into zotero-worksheet-cli
seonghobae Sep 4, 2026
70afe20
Merge current receipt repair parent into zotero-worksheet-cli
seonghobae Sep 4, 2026
6b775d6
Merge remote-tracking branch 'origin/autoresearch/zotero-steward-revi…
seonghobae Sep 4, 2026
dd391e5
merge(zotero): adopt current worksheet parent
seonghobae Sep 4, 2026
0f5002e
merge(zotero): adopt current worksheet parent and gap baseline
seonghobae Sep 5, 2026
1b5dfde
merge(research): inherit verified source and proposal approval binding
seonghobae Sep 5, 2026
d8d9928
merge(research): inherit canonical local transport repairs into PR #26
seonghobae Sep 5, 2026
56f3f1d
merge(research): inherit deterministic transport framing regression i…
seonghobae Sep 5, 2026
f60c07f
merge(zotero): propagate validated approval ordering through PR 26
seonghobae Sep 5, 2026
e2dc600
merge(research): inherit bounded metadata reads into PR #26
seonghobae Sep 6, 2026
227b3e9
merge(research): retain export pair and verified worksheet scope
seonghobae Sep 6, 2026
68575a6
test(research): reproduce output unlink and implicit flush races
seonghobae Sep 6, 2026
ab391b2
fix(research): preserve failed outputs without implicit write retries
seonghobae Sep 6, 2026
fb5b5e5
test(research): expose canonical output alias collision before reads
seonghobae Sep 6, 2026
e928858
fix(research): reject canonical artifact aliases before snapshot read
seonghobae Sep 6, 2026
58ff598
test(research): cover both output admission failures
seonghobae Sep 6, 2026
7ad6386
docs(research): record private export failure and alias evidence
seonghobae Sep 6, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ All notable changes to ConceptWeave are documented here.
- Owner-only file permissions for sensitive local Zotero classification reports.
- A complete-review evaluator that rejects partial steward labels as full reclassification evidence.
- A snapshot-bound steward worksheet with one blank decision per bibliographic item and no duplicated bibliographic text.
- An explicit `--worksheet` CLI mode that writes the live worksheet with owner-only report protections.

### Security

Expand Down
202 changes: 192 additions & 10 deletions crates/conceptweave-zotero/src/main.rs
Original file line number Diff line number Diff line change
@@ -1,12 +1,79 @@
#![forbid(unsafe_code)]
#![cfg_attr(coverage_nightly, feature(coverage_attribute))]

use conceptweave_zotero::read_local_snapshot;
use conceptweave_zotero::{build_steward_review_worksheet, read_local_snapshot};
use std::env;
use std::fs::{self, File, OpenOptions};
use std::io::{self, BufWriter, Write};
use std::path::{Path, PathBuf};

const USAGE: &str = "usage: conceptweave-zotero /tmp/REPORT.json | --worksheet /tmp/REPORT.json /tmp/WORKSHEET.json";

fn parse_output_request<I, S>(args: I) -> Result<(Option<String>, String), &'static str>
where
I: IntoIterator<Item = S>,
S: Into<String>,
{
let mut args = args.into_iter().map(Into::into);
let first = args.next().ok_or(USAGE)?;
let request = if first == "--worksheet" {
let report = args
.next()
.ok_or("--worksheet requires report and worksheet output paths")?;
let worksheet = args
.next()
.ok_or("--worksheet requires report and worksheet output paths")?;
if report == worksheet {
return Err("report and worksheet output paths must differ");
}
(Some(report), worksheet)
} else {
(None, first)
};
if args.next().is_some() {
return Err("unexpected extra argument");
}
Ok(request)
}

fn validate_output_request(
report_output: Option<&str>,
output: &str,
) -> io::Result<(Option<PathBuf>, PathBuf)> {
let output = validate_output_path(output)?;
let report_output = report_output.map(validate_output_path).transpose()?;
if report_output.as_ref() == Some(&output) {
return Err(io::Error::new(
io::ErrorKind::InvalidInput,
"report and worksheet output paths must differ",
));
}
Ok((report_output, output))
}

fn write_private_output(path: &Path, content: &[u8]) -> io::Result<()> {
write_private_output_with(path, content, write_all_and_flush)
}

#[cfg_attr(coverage_nightly, coverage(off))]
fn write_all_and_flush(writer: &mut BufWriter<File>, content: &[u8]) -> io::Result<()> {
writer.write_all(content)?;
writer.flush()
}

fn write_private_output_with(
path: &Path,
content: &[u8],
write: fn(&mut BufWriter<File>, &[u8]) -> io::Result<()>,
) -> io::Result<()> {
let file = create_report_file(path)?;
let mut writer = BufWriter::new(file);
let result = write(&mut writer, content);
// Never retry buffered writes on drop or unlink a possibly replaced pathname.
let _ = writer.into_parts();
result
}

#[cfg_attr(coverage_nightly, coverage(off))]
/// Returns canonical directories in which a sensitive report may be created.
fn allowed_output_parents() -> Vec<PathBuf> {
Expand Down Expand Up @@ -97,25 +164,140 @@ fn create_report_file_with(
#[cfg_attr(coverage_nightly, coverage(off))]
/// Reads one Zotero snapshot and writes its sensitive local proposal report.
fn main() -> Result<(), Box<dyn std::error::Error>> {
let output = env::args()
.nth(1)
.ok_or("usage: conceptweave-zotero /tmp/OUTPUT.json")?;
let output = validate_output_path(&output)?;
let (report_output, output) = parse_output_request(env::args().skip(1))?;
let (report_output, output) = validate_output_request(report_output.as_deref(), &output)?;
let report = read_local_snapshot()?;
if report.zotero_version.starts_with("9.") {
eprintln!("Zotero 9 Local API is read-only; writing a local proposal report only");
eprintln!("Zotero 9 Local API is read-only; writing local proposal output only");
}
if let Some(report_output) = report_output {
let worksheet = build_steward_review_worksheet(&report)?;
let report_content = serde_json::to_vec_pretty(&report)?;
let worksheet_content = serde_json::to_vec_pretty(&worksheet)?;
write_private_output(&report_output, &report_content)?;
write_private_output(&output, &worksheet_content)?;
} else {
write_private_output(&output, &serde_json::to_vec_pretty(&report)?)?;
}
let file = create_report_file(&output)?;
let mut writer = BufWriter::new(file);
serde_json::to_writer_pretty(&mut writer, &report)?;
writer.flush()?;
Ok(())
}

#[cfg(test)]
mod tests {
use super::*;

#[test]
fn worksheet_mode_is_explicit_and_rejects_ambiguous_arguments() {
let report = "/tmp/conceptweave-zotero-report.json";
let worksheet = "/tmp/conceptweave-zotero-worksheet.json";
assert_eq!(
parse_output_request(vec!["--worksheet", report, worksheet]),
Ok((Some(report.to_owned()), worksheet.to_owned()))
);
assert_eq!(
parse_output_request(vec![report]),
Ok((None, report.to_owned()))
);
assert_eq!(parse_output_request(Vec::<&str>::new()), Err(USAGE));
assert!(parse_output_request(vec!["--worksheet"]).is_err());
assert!(parse_output_request(vec!["--worksheet", report]).is_err());
assert!(parse_output_request(vec![report, "extra"]).is_err());
assert!(parse_output_request(vec!["--worksheet", report, report]).is_err());
}

#[test]
fn canonical_output_aliases_are_rejected_without_creating_files() {
let output =
Path::new("/tmp").join(format!("conceptweave-alias-{}.json", std::process::id()));
let canonical = output
.parent()
.unwrap()
.canonicalize()
.unwrap()
.join(output.file_name().unwrap());
let result =
validate_output_request(Some(output.to_str().unwrap()), canonical.to_str().unwrap());
assert!(!output.exists());
assert_eq!(result.unwrap_err().kind(), io::ErrorKind::InvalidInput);
let other = unique_temp_path("other-output");
assert!(
validate_output_request(Some(output.to_str().unwrap()), other.to_str().unwrap())
.is_ok()
);
assert!(validate_output_request(None, output.to_str().unwrap()).is_ok());
assert!(validate_output_request(None, "relative-output").is_err());
assert!(
validate_output_request(Some("relative-report"), output.to_str().unwrap()).is_err()
);
}

#[test]
fn write_failure_preserves_replacement_at_output_path() {
let output = unique_temp_path("write-race");
let retained = unique_temp_path("write-race-retained");
assert!(!output.exists());
assert!(!retained.exists());
let error = write_private_output_with(&output, b"content", |_, _| {
let output = unique_temp_path("write-race");
fs::rename(&output, unique_temp_path("write-race-retained"))?;
let mut replacement = OpenOptions::new()
.write(true)
.create_new(true)
.open(output)?;
replacement.write_all(b"unrelated replacement")?;
Err(io::Error::new(io::ErrorKind::WriteZero, "injected failure"))
})
.unwrap_err();
let replacement = fs::read(&output);
let _ = fs::remove_file(output);
fs::remove_file(retained).unwrap();
assert_eq!(error.kind(), io::ErrorKind::WriteZero);
assert_eq!(replacement.unwrap(), b"unrelated replacement");
}

#[test]
fn write_failure_does_not_retry_buffered_bytes_on_drop() {
let output = unique_temp_path("write-buffer");
let retained = unique_temp_path("write-buffer-retained");
assert!(!output.exists());
assert!(!retained.exists());
let error = write_private_output_with(&output, b"buffered content", |writer, content| {
fs::rename(
unique_temp_path("write-buffer"),
unique_temp_path("write-buffer-retained"),
)?;
writer.write_all(content)?;
Err(io::Error::new(io::ErrorKind::WriteZero, "injected failure"))
})
.unwrap_err();
let retained_bytes = fs::read(&retained).unwrap();
fs::remove_file(retained).unwrap();
assert_eq!(error.kind(), io::ErrorKind::WriteZero);
assert!(retained_bytes.is_empty());
}

#[test]
fn failed_private_output_is_preserved_and_requires_a_new_path() {
let output = unique_temp_path("failed-output");
let _ = fs::remove_file(&output);
let error = write_private_output_with(&output, b"content", |_, _| {
Err(io::Error::new(
io::ErrorKind::WriteZero,
"injected write failure",
))
})
.unwrap_err();
assert_eq!(error.kind(), io::ErrorKind::WriteZero);
assert_eq!(fs::metadata(&output).unwrap().len(), 0);
assert!(write_private_output(&output, b"retry").is_err());
fs::remove_file(&output).unwrap();

write_private_output(&output, b"complete").unwrap();
assert_eq!(fs::read(&output).unwrap(), b"complete");
assert!(write_private_output(&output, b"replacement").is_err());
fs::remove_file(output).unwrap();
}

fn unique_temp_path(suffix: &str) -> PathBuf {
env::temp_dir().join(format!(
"conceptweave-zotero-{}-{suffix}.json",
Expand Down
1 change: 1 addition & 0 deletions docs/TRD.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@ Provider deserialization captures each complete JSON object before projecting me
The complete metadata-review evaluator rejects unequal label cardinality or nonempty `pending_source_item_keys` with `IncompleteReview` before governance. The shared evaluator then recomputes the complete inventory and pending ancestry, so clearing pending keys and rewriting the proposal digest still fails local validation. Because shared validation rejects blank, duplicate, and unknown keys, equal cardinality proves bibliographic label coverage. Sampled evaluation still supports pending sources; completion does not prove a Zotero mutation or full-text approval.
A successful classification report carries an `audit_summary` whose snapshot, bibliographic, proposed-disposition, provenance-complete, abstention, duplicate-candidate, failure, and per-disposition counts are derived from the same in-memory immutable snapshot. Zotero item version zero remains a valid observed coordinate for never-synced Zotero 9 records; provenance completeness rejects a missing item key rather than inventing a positive-only version invariant. Reader failures return an error instead of a partial report; therefore a returned report records `failure_count=0` rather than hiding partial failures.

`conceptweave-zotero --worksheet /tmp/REPORT.json /tmp/WORKSHEET.json` validates both canonical destinations and rejects aliases before reading one live snapshot. Both artifacts are built and serialized before the first file write. Writes are sequential, not an atomic publication. Any write failure propagates without pathname deletion or implicit buffer-flush retry. A complete report and an empty/partial worksheet may remain; that is not a completed pair. Inspect retained owner-only files and use new output paths for another capture. Never automatically overwrite or infer approval from a surviving artifact. A successful flush is not a crash-durability guarantee.
The review worksheet is a deterministic item-key-ordered projection of the report. It binds library/rule revisions, raw-snapshot digest, required `proposal_digest` from the existing v2 scope hash, complete item coordinates, proposal, abstention reason, and one initially empty decision per bibliographic item. Construction reuses `validate_classification_report`, mapped to `WorksheetError::InvalidReport`, before worksheet-specific nonblank identity and abstention checks. This avoids a second drifting audit implementation while admitting structurally valid pending source evidence. It deliberately omits bibliographic text and matched evidence; stewards consult the owner-only report by item key. Missing proposal binding fails deserialization. Subsequent progress, application and finalization owners must compare this field to the recomputed report binding; present-but-blank or rewritten values are not authority. Legacy worksheets must be regenerated, and independent approval remains separate.

The local report can contain titles, tags, matched metadata, and abstention abstracts. It is sensitive steward-review material, remains outside the repository, and is not a publication artifact.
Expand Down
2 changes: 2 additions & 0 deletions docs/adr/0006-zotero-research-intake.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@

## Context

Proposed September 7 export-failure amendment: when exporting sensitive report/worksheet pairs, a failed write may race with pathname replacement, and destroying a buffered writer may write pending bytes after failure. We choose to preserve artifacts, disassemble the existing buffer without flushing, and propagate the original error, rejecting pathname cleanup or an automatic retry. This extends the existing private-creation policy to the common output writer and the pair's second-file failure. The cost is retained empty/partial files and operator inspection; sequential export is not a transaction or crash-durable publication. Both canonical destinations must differ before the Local API read, including aliases such as `/tmp` and `/private/tmp`. RED `68575a6` proves replacement deletion and implicit drop-flush; `ab391b2` removes both behaviors. RED `fb5b5e5` proves alias collision; `e928858` rejects it before snapshot capture. Later CLI owners must inherit this implementation instead of restoring cleanup. No Zotero mutation or approval authority follows from local output.

CWL needs a reproducible inventory of ontology research without turning keyword matches into authoritative library organization. The current desktop is Zotero 9.0.6, whose Local API supports reads but not writes. Zotero documents Local API writes only for Zotero 10+, where they require user-granted authorization and `Zotero-Server-ID`; this slice therefore has no mutation capability. The library is mutable while pagination is in progress, duplicate metadata does not prove that two records should be merged, and the local report contains titles and item keys that must not be written into the repository.

Zotero's Local API documentation states that production clients should request `Zotero-API-Version: 3`; the response exposes `Zotero-API-Version` and `Zotero-Schema-Version`. The API version is the compatibility contract. The schema version is therefore recorded and required to remain stable across the snapshot, but it is not hard-coded to the developer workstation's current schema 42 because Zotero can legitimately revise the local data schema while retaining API v3 compatibility.
Expand Down
10 changes: 10 additions & 0 deletions docs/product-technical-gap-baseline.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,16 @@ This file records code-current product and technical gaps. Exact PR/check/run co

## September 6 source inventory checkpoint

### September 7 PR26 private export repair

Final source `58ff5985890d5a0b4aaadaa1f8d604e1bc96a1e2` passes 163 tests/24 suites including three doctests, strict Clippy, warnings-denied rustdoc, formatting, CI contract and diff checks. The unchanged pinned coverage gate passes 294/294 functions, 2,536/2,536 normalized regions and 422/422 normalized branches. Raw coverage remains 3,382/3,444 lines, 5,139/5,246 regions and 377/422 branches, not 100%. Logs: `/tmp/conceptweave-pr26-verified.log` and `/tmp/conceptweave-pr26-{clippy,rustdoc,coverage}-verified.log`. The earlier coverage failure is retained in `/tmp/conceptweave-pr26-coverage.log`; it is not a successful checkpoint.

Original PR26 `e2dc6006ed3e56d8388e82912826cf37efed0541` passed 128 tests/24 suites. Ordinary merge `227b3e9` retains it and PR25 `51631fbf711b403a40f3b9fafa2ec3958d54ceaf`; integrated tests passed 160/24. RED `68575a6` compiled and failed both regression tests: a replacement file was deleted after write failure, and buffered bytes were flushed while dropping a failed writer. `ab391b2` preserves the explicit write result, disassembles the buffer without retry, and removes pathname cleanup from both the shared writer and second-artifact failure. Existing private creation, successful write and overwrite-refusal checks remain.

RED `fb5b5e5` compiled and failed canonical-alias admission. `e928858` rejects equal canonical destinations before the Local API read; both artifacts are still serialized from one captured report before either is written. Independent read-only review found no further production defect. The first coverage run exposed two untested error propagation points in the extracted admission function; `58ff598` adds both invalid-path cases without changing runtime or coverage exclusions. Final verification is recorded separately below.

TRD and Proposed ADR 0006 describe retained partial files and sequential, nontransactional output. A report surviving a failed worksheet write is not a completed pair, approval or durable publication. Root and later CLI owners still require this inherited fix; PR27 finalization must also compare the newly required worksheet proposal binding before governance. Actual decisions and independent approvals remain 0/3,715 plus four unresolved standalone sources. No real source metadata or authority was created for these synthetic tests. Native Visual Inspection was attempted but the Mac is locked, so no new screen evidence exists. No hosted GREEN, protected approval/merge or release is claimed.

### PR25 worksheet admission and identity repair

Original PR25 `c6b4c17e931951a2e1d4ea79ac79363f6306a5bf` passed 126 tests/24 suites. Normal merge `4a1a3bb` retains it and PR24 `35c57ca4510a65cf48069285d78b95cf47db65ba`; integrated tests passed 153/24. RED `900038e` compiled with 4 passing and 2 failing tests: omitted retained inventory and hidden pending keys still produced worksheets. `5b54d06` reuses shared report validation and removes 54 lines of divergent audit/coordinate checks. Valid standalone, orphan, cyclic and attached source metadata remains reviewable with blank decisions. Completion admission remains distinct.
Expand Down