Map: Storage-object deposit model

[johnf]

Destination

A PR on a branch of this repo containing the openapi.yaml changes and substantial docs pages that specify the storage-object deposit model (session-based create → stage → finalise, backend-defined materialisation), pitched at ~90% Technology Committee-ready.

Notes

• Execution is in scope for this map (overriding wayfinder's plan-only default): the destination is the PR itself, so the final tickets draft the spec and docs, not just decisions.
• Domain: RO-Crate catalog API (read-only today: entities, /entity/{id}/rocrate, search, files). The v1 read–write discussion paper (no local copy on disk — the .scratch/ copy from the charting session is gone) proposed entity-granular writes; the committee pushed back because one crate spans many entities (collections → subcollections → items → files/MediaObjects).
• Core model: a storage object is the real thing on disk — an RO-Crate plus all files it references. Deposit is a session: create (returns UID) → stage crate + files → finalise. The backend then materialises catalog entities from it via implementation-specific rules (PARADISEC: one item + files; LDaCA: exploded collections/subcollections/entities). Materialisation can be many-to-one: two storage objects can contribute to a single merged entity (e.g. a shared speaker).
• Skills for ticket sessions: /grilling and /domain-modeling for HITL tickets.
• Australian spelling throughout docs.
• Discussion asset for Slack and the Thursday 2026-07-23 tech meeting: slack-discussion-points.md at the repo root (local, untracked) — all five open points (conformance placement, entity lifecycle on re-deposit, curation-crate enrichment, constrained entity DELETE, update-deposit concurrency). Supersedes the former thursday-agenda.md, now deleted.

Decisions so far

Settled during the charting session on 2026-07-21 (full detail in the "Charting-session decisions" comment below):

• Destination is a PR, not a paper v2 — spec + docs changes in this repo; committee expected to approve with small tweaks.
• Storage object is the sole write pathway — no entity CRUD, no standalone file endpoints; entities are backend-materialised projections.
• Storage objects are first-class readable resources — bidirectional linkage; GET /storage-object/{id}/crate returns the original deposited crate (entity-level crates may be virtualised sub-views).
• Entity lifecycle on re-deposit is implementation-defined — pruning can't be mechanical because materialisation may merge entities across storage objects; to be reviewed at Thursday's meeting per the agenda asset.
• Entity→storage-object link is plural — storageObjectIds array: "the storage objects whose current versions contribute to this entity".
• Carry-overs from the v1 paper adopted as settled — ID-minting duality, two file-upload modes, coarse write scope, implementation-defined validation with uniform 422, last-write-wins concurrency, implementation-defined tombstones. Dead: §2 Alternatives A/B, entity PATCH, standalone file endpoints, client-driven entity DELETE.
• Session lifecycle and endpoint shapes — deposit is a top-level resource (POST /deposits create / POST /storage-object/{id}/deposits update, no intent flag); staging via PUT /deposit/{id}/crate and PUT /deposit/{id}/file/{fileId} (fileId = crate @id, any order); state machine open → finalising → complete, validation failure returns to open, DELETE aborts, implementation-defined expiry; one storage object per deposit.
• Update and versioning flow — crate-as-manifest carry-forward: update deposit stages only changed files; at finalise the new crate resolves per file entity staged → carried-forward-by-@id → permitted-dangling, absent files drop out; baseline pinned at deposit creation, finalise replaces wholesale (last finalise wins as a unit, optional 409 on stale finalise); current-version-only surface confirmed.
• Finalise semantics — sync or async at server discretion (200 complete / 202 finalising, poll GET /deposit/{id}, no capability flag); failure atomicity mandated, success-visibility implementation-defined (complete ⇒ everything readable); one violations shape (422 sync, errors field on deposit async), non-validation failures also return to open; finalise verifies presigned uploads (no per-file complete call, per-file staging status on the deposit); thin deposit response — materialised-entity list lives on the storage-object read surface.
• Storage-object read surface — visibility implementation-defined, docs recommend choosing public-with-access-object or depositor-only (pros/cons of both); minimal GET /storage-objects list; lean GET /storage-object/{id} body (id, entityIds, timestamps, access — crate is sole content inventory); optional singular storageObjectId on File; GET /entity/{id}/rocrate re-specced as implementation-defined provenance (stored or virtualised view) but always a valid RO-Crate about the entity.
• Storage-object deletion — plain DELETE /storage-object/{id}, no mandated preconditions (implementation-defined 409-with-reason); entity knock-on implementation-defined (deliberate retention legitimate); enrichment via curation storage objects instead of entity UPDATE; constrained DELETE /entity/{id} for contributor-less entities only (narrows the dead entity-DELETE carry-over; both on Thursday's agenda); one 410/404 tombstone policy per implementation covering entities too, with a Tombstone schema and a capabilities flag; delete wins over open deposits (finalise fails back to open); ID reuse permitted, recreation is a new object; delete-preview deferred to a docs recipe over entityIds + storageObjectIds.
• Capabilities and discovery — the whole deposit surface (session endpoints, storage-object read surface, entity/file linkage fields) is one registered extension deposit, discovered by key presence in capabilities.extensions; DepositCapability requires idMinting (client/server/both), fileUpload array (inline/presigned) and tombstonePolicy ("410"/"404", per the deletion hand-off), with optional depositTtlSeconds/maxFileSizeBytes; deliberately excluded: finalise mode, visibility model, auth/scopes (OpenAPI security + docs prose), concurrency flag (pending Thursday); conformance placement (mandatory / core-but-optional / extension) to be socialised on Slack — asset: slack-discussion-points.md at the repo root.
• Draft the openapi.yaml changes — drafted on branch deposit-extension-spec (openapi.yaml + CHANGELOG.md): all deposit-extension endpoints, schemas, capabilities, write scope and 409/410/422 errors, spec bumped to 0.3.0 with changelog entry; Redocly lint and the full docs build pass; docs pages and the PR continue on that branch.
• Write the docs pages — written on deposit-extension-spec as an Extensions subcategory (docs/extensions/deposit/): the extension guide (model, materialisation examples, capability, visibility patterns) plus Depositing / Updating / Deletion & Lifecycle flow pages; registry table and capability examples updated; build, link check and lint pass.
• Assemble the PR — destination reached: draft PR #32 opened from deposit-extension-spec with the model summary, decision-ticket links and the two open committee points; lint and full build re-verified before pushing. No open tickets remain — the residual fog (Thursday's meeting, the Slack conformance thread) resolves as small tweaks on the PR or falls out of scope as post-review iteration.

Not yet specified

• Whatever Thursday's tech meeting (2026-07-23) feeds back on the entity-lifecycle point, the update-deposit concurrency model (optional stale-finalise 409 vs single-open-deposit lock vs capability-signalled — if capability-signalled, DepositCapability gains a field), the curation-crate enrichment pattern, and the constrained entity DELETE — may adjust spec wording. Likewise the Slack thread on deposit's conformance placement (mandatory / core-but-optional / extension; extension is drafted).
• Bulk/batch deposit ergonomics beyond a single storage object per session (deferred in v1 paper; may stay deferred).

Out of scope

• Exposing storage/file version history through the API (v1 paper §6.4 — implementation detail, e.g. OCFL; current-version-only surface).
• Backend implementation work in PARADISEC / LDaCA archives.
• The committee approval process itself and any post-review iteration on the PR.