Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
f9ec379
test(zotero): require bound full-text capture admission
seonghobae Sep 5, 2026
9aafff5
test(zotero): reproduce environment proxy routing of local requests
seonghobae Sep 5, 2026
a2848e5
fix(zotero): bypass environment proxies for both local agents
seonghobae Sep 5, 2026
3d4e2b4
test(zotero): specify full-text content and provenance capture
seonghobae Sep 5, 2026
9925a2e
fix(zotero): integrate direct local transport regression repair
seonghobae Sep 5, 2026
39b7fe8
test(zotero): activate failing full-text capture contracts
seonghobae Sep 5, 2026
5f36ff5
feat(zotero): retain report-bound full-text observations
seonghobae Sep 5, 2026
53efefb
test(zotero): exercise full-text replay and admission failures
seonghobae Sep 5, 2026
bcf787e
fix(zotero): expose the capture command in usage
seonghobae Sep 5, 2026
f3a2847
test(zotero): verify full-text HTTP transport boundaries
seonghobae Sep 5, 2026
f7d3530
test(zotero): require replay admission before digest allocation
seonghobae Sep 5, 2026
301d9d5
fix(zotero): bound replay before streaming its content digest
seonghobae Sep 5, 2026
2d9ec5c
test(zotero): reproduce shared exact-body-limit rejection
seonghobae Sep 5, 2026
c959505
fix(zotero): enforce inclusive response byte limits consistently
seonghobae Sep 5, 2026
f424173
refactor(zotero): keep private writes on one tested dispatch path
seonghobae Sep 5, 2026
eb3d282
docs(zotero): define private full-text capture and approval boundaries
seonghobae Sep 5, 2026
d24a74a
test(zotero): reject a missing report schema at the transport boundary
seonghobae Sep 5, 2026
847535d
test(zotero): reject oversized replay counts and valid-JSON tampering
seonghobae Sep 5, 2026
b6ff334
fix(zotero): integrate inclusive response bounds and wire regressions
seonghobae Sep 5, 2026
7854b3a
test(zotero): require late and invalid-clock sweeps to fail
seonghobae Sep 5, 2026
2c2226f
fix(zotero): verify clock and request failure propagation
seonghobae Sep 5, 2026
733425d
refactor(zotero): name the shared private output writer type
seonghobae Sep 5, 2026
9cc5bef
docs(zotero): record retained-text KPI and exact verification evidence
seonghobae Sep 5, 2026
e19d95f
docs(zotero): link the capture successor and current stack gates
seonghobae Sep 5, 2026
7cfa8d7
docs(research): bind owner audits to source license and release evidence
seonghobae Sep 5, 2026
3ae0bbf
docs(research): keep transient subgroup evidence separate from retain…
seonghobae Sep 5, 2026
2402604
docs(zotero): verify fulltext provider gap on current upstream source
seonghobae Sep 5, 2026
53e0bef
docs(zotero): record canonical authenticated transport regressions an…
seonghobae Sep 5, 2026
895e64c
docs(governance): record base-ref discrepancy without assuming its cause
seonghobae Sep 5, 2026
75da75c
Merge canonical Zotero transport repairs into full-text capture
seonghobae Sep 5, 2026
a10de57
docs: record canonical transport cascade and exact verification
seonghobae Sep 5, 2026
1e7d23c
docs: advance next action after completed transport cascade
seonghobae Sep 5, 2026
6b33c07
Merge verified private artifact boundary repair into PR 36
seonghobae Sep 5, 2026
87f02a9
merge(zotero): propagate validated approval ordering through PR 36
seonghobae Sep 5, 2026
d3f991d
merge(research): inherit bounded metadata reads into PR #36
seonghobae Sep 6, 2026
707f2f2
merge: retain full text capture with reviewed source scope foundation
seonghobae Sep 6, 2026
c7052d9
test: verify full text source continuity through shared admission
seonghobae Sep 6, 2026
f26ce5b
docs: trace capture source scope and inherited private failure policy
seonghobae Sep 6, 2026
aca72e7
docs: record PR36 source capture verification and open campaign gaps
seonghobae Sep 6, 2026
2cbac82
test(zotero): require manifest library version binding
seonghobae Sep 6, 2026
2c16fea
fix(zotero): bind manifest bookends to library version
seonghobae Sep 6, 2026
4c1f81c
test(zotero): bind manifest fixture to library version
seonghobae Sep 6, 2026
9be1ca0
test(zotero): emit manifest library version on wire
seonghobae Sep 6, 2026
3ab0041
revert(zotero): preserve observed local full-text contract
seonghobae Sep 6, 2026
4cfff3f
fix(zotero): bound persisted full-text capture size
seonghobae Sep 6, 2026
4d22595
docs(zotero): document persisted capture and partial-write boundaries
seonghobae Sep 6, 2026
e517fc0
docs(test): cover serialized full-text capture ceiling
seonghobae Sep 6, 2026
9fe952e
test(zotero): exercise restored capture persisted boundary
seonghobae Sep 7, 2026
308dc31
fix(zotero): share persisted capture verification boundary
seonghobae Sep 7, 2026
51c2837
docs(test): record shared capture verifier boundary evidence
seonghobae Sep 7, 2026
9392938
docs(gap): distinguish verifier repair and visual evidence
seonghobae Sep 7, 2026
c2c1558
docs(trd): specify persisted capture ceiling
seonghobae Sep 7, 2026
5b07854
docs(gap): record capture verifier workspace gates
seonghobae Sep 7, 2026
c062845
docs(gap): record terminal capture verifier coverage
seonghobae Sep 7, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ ConceptWeave owns automatic, evidence-bound **Semantic Model Engineering**. Do n
- No direct cross-service application-table SQL.
- New database objects, when introduced, use descriptive two-or-more-word `snake_case` names and 3NF by default.
- Preserve source evidence, truth status, and publication state separately.
- Keep Zotero full-text captures separate from metadata reports and approval receipts; restored captures require bounded verification, and local HTTP continuity is not peer authentication.
- Published semantic truth is immutable; correction uses supersession/new release.
- Public Rust APIs require beginner-readable documentation.
- Owned production coverage target is 100% line/function/region/branch where tooling exposes it.
Expand Down
2 changes: 2 additions & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ flowchart LR

## DDD context map

Research Intake's Zotero adapter retains optional full-text observations in a separate private artifact bound to the original metadata report. It remains inside ConceptWeave: acquisition is supporting evidence work, not a publication authority or another research system of record. Provider version counters remain opaque at this Anti-Corruption Layer; downstream classification and review must explicitly adopt new content under fresh evidence bindings. See [ADR 0006](docs/adr/0006-zotero-research-intake.md).

| Context | Type | Owns | Does not own |
| --- | --- | --- | --- |
| Source Observation | Supporting | immutable observations, parser receipts, evidence locations | source-system business truth |
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ All notable changes to ConceptWeave are documented here.

### Added

- Private, replayable paper-text capture for later research review, preserving unavailable material and leaving earlier reports and approvals unchanged.

- Full-library research-source audit separating available text, incomplete indexing and missing material from reviewed classification; no paper is excluded because its abstract or text is unavailable.
- Initial ConceptWeave product, DDD, security, test, and operability baselines.
- Rust 1.98.0 `conceptweave-domain` foundation with evidence-bound semantic candidate contracts.
Expand All @@ -37,7 +39,13 @@ All notable changes to ConceptWeave are documented here.

### Security

- Local research requests bypass environment-configured proxies. This prevents unintended proxy forwarding; local peer authentication remains an explicit release limitation.

- Source receipts bind complete captured metadata and actual classifier inputs; earlier report and review artifacts require regeneration under the versioned digest representation.
- Golden-set evaluation rejects changed predictions or evidence under an earlier approval. Proposal-bound approvals must be reissued; aggregate receipts identify the actual evaluated proposal run.
- Model-generated semantics remain non-authoritative until deterministic validation and authorized review.
- Unsafe Rust is forbidden in the core domain crate.

### Fixed

- Research reads accept valid responses exactly at their documented size limit while still rejecting oversized, incomplete or invalidly encoded responses.
2 changes: 2 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,5 @@ Follow `AGENTS.md`, `ARCHITECTURE.md`, accepted ADRs, and the organization maste
ConceptWeave's core invariant is: **inference is not authority**. Every generated concept, relation, constraint, dimension, measure, or physical mapping must retain evidence and pass the explicit governance lifecycle before publication.

Keep domain logic in bounded domain modules, LLM/provider logic behind ports/adapters, and source/consumer systems independent. Prefer deterministic validation and explicit abstention over plausible unsupported output.

Zotero source capture must not alter the metadata report or renew its approval. Preserve private-file protections and the full bibliographic denominator, including missing and partial text.
8 changes: 8 additions & 0 deletions OPERABILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,11 @@ ConceptWeave has no production network service or durable database in the founda
- downstream catalog unavailable: publication retains a durable release/outbox receipt and does not lose the governed release.

Concrete SLO/RPO/RTO values require measured runtime evidence and are not guessed in the foundation.

## Local paper-text capture

The proposed `--capture-full-text` command uses an existing private metadata report and creates a separate owner-only file. Keep the original report: the new file cannot replace it, renew review, or prove that all text came from one atomic snapshot. No Zotero authorization prompt, mutation or model request is part of this command.

A changed library, missing provider identity, unexpected response, malformed text or exhausted budget rejects the run. Preserve earlier artifacts; do not disable the checks or overwrite an old file to retry. If metadata has changed, capture a new report and start a separately bound review campaign. Otherwise investigate the reported boundary and rerun to a new temp path. An expected missing-text response is retained. Admission failures create no output, while a write or flush failure may leave a private partial file at the create-new path; the writer deliberately does not unlink or retry buffered bytes against a pathname that may have been replaced.

Responses are limited to 8 MiB each and 256 MiB total. The persisted compact JSON capture has a separate 512 MiB ceiling because nested source JSON can expand when its text is escaped by the outer capture envelope. Capture verification counts the exact serialized representation before a newly acquired capture can be returned, so the CLI does not create an artifact that the bounded restore path is specified to reject solely because of JSON-encoding expansion. This preflight streams into a counting writer and does not allocate a second encoded copy. The sweep also has a five-minute admission/completion limit and finite local request timeouts. Source text stays in memory until capture completes; hashing and final writing stream. The CLI deliberately does not delete prior reports or schedule private-file cleanup. Review retention according to the research library's policy, and never attach these files to a public PR.
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,14 @@ cargo doc --workspace --no-deps

The repository CI also validates the JSON Schema, lock/toolchain freshness, documentation contracts, and coverage expectations defined by the current source.

For the proposed local research intake, preserve available paper text separately from an existing private report:

```bash
cargo +1.98.0 run --locked -p conceptweave-zotero -- --capture-full-text /tmp/REPORT.json /tmp/CAPTURE.json
```

Zotero 10+ must be running. The report must be an unchanged owner-only file from that library; the capture path must be a new file directly in the system temp directory. Missing or partial text stays visible, and the command does not classify papers, approve decisions or modify Zotero. Keep both files private; see [operation and retry limits](OPERABILITY.md).

## Core contract

A semantic candidate is not the same thing as published semantic truth.
Expand Down
2 changes: 2 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ All source artifacts, generated candidate payloads, external ontology files, mod

Local Zotero classification reports can contain bibliographic titles, tags, matched values, and abstention abstracts. They are sensitive review material, remain outside the repository, and are not publication artifacts.

Full-text captures additionally retain exact paper text and attachment metadata. They use new `0600` local files, static transport errors and no content logging. Both local HTTP agents explicitly disable inherited proxies; redirects remain disabled. Replay limits record and response sizes before parsing/hash work, and unchanged hashes do not authenticate a replaced local artifact or the provider. Full-text source strings remain untrusted data, never instructions. Metadata approval cannot authorize newly captured text.

## Required controls

- source size, type, nesting, archive/decompression, and parser-time bounds;
Expand Down
6 changes: 6 additions & 0 deletions TEST_STRATEGY.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@
- lockfile freshness and clean-tree verification;
- public Rust documentation with `missing_docs` denied.

## Local research capture regressions

The full-text suite covers report admission, exact response retention, parent/item revision binding, independent content versions, missing/empty/partial text, duplicate/foreign manifest rejection, bookend drift, byte/deadline boundaries and replay under changed or recomputed digests. It also distinguishes the 256 MiB aggregate source-body budget from the 512 MiB persisted compact-JSON ceiling: an escape-heavy valid capture fixture stays within a proportionally scaled raw-body budget while its outer JSON grows past the corresponding persisted ceiling, and the size validator rejects that representation while accepting the exact serialized-byte boundary. Synthetic HTTP tests cover headers, network/redirect failures, strict encoding and response limits without touching the running Zotero library. Proxy isolation uses fresh subprocess environments for all six supported proxy variable spellings across the three existing local transport paths. Synthetic text is only a unit/integration fixture; live aggregate evidence is separately recorded in doctoring and never reported as approved labels.

The shared restored-capture verifier also has an exact compact-JSON boundary regression: the existing valid report/capture fixture passes at its serialized byte count and returns the budget error with one byte less allowance. The private limit parameter exposes the real verifier error path without allocating a 512 MiB fixture; the public verifier retains its fixed production ceiling and report-before-size validation order. This scaled test is not live-library or approval evidence.

## Future product test families

### Source observation
Expand Down
Loading