Repository navigation
feat: make generated payload ownership explicit - #146
Merged
Merged
Conversation
Tiled FileFormat reads wrote their payloads with no owner, so the same layer could not be read a second time: the payloads from the first read refused the write (LAS016, COPC007). Layer authoring also built content in scratch stages that loaded every payload and borrowed file identities to anchor them, which made tiled COPC reads warn "Could not open asset". - Build layer content in in-memory stages opened with LoadNone and export payloads from in-memory layers; drop the layer identifier rewriting. - Add a layer-level AuthorPointCloudTiledAssetWithPayloads overload with typed diagnostics and move the COPC adapter onto it. - Record payload ownership per layer identity in payload-owner-<hash>.manifest: a read replaces the payloads its layer generated before, removes superseded ones, and refuses any other file. Ownership is claimed before the first write and restored on failure. - Apply the same rule when materializing a generated-cache hit, taking over files that already hold the cached bytes. - Report the reason behind tiled author failures in LAS, LAZ, and COPC. - Fix the stream benchmark build on macOS. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The review of the ownership record found that sharing one flat set of payload names between layers let a failed re-read delete payloads a root still referenced, let concurrent reads and stale or adopted records reach another layer's files, collapsed near-equal arguments onto one owner, and required writes on every cache hit. - Publish owned payloads in <payloadDirectory>/<owner>/<generation>/. The owner hashes the source, relative to the payload directory for local files, and the layer's exact file-format arguments, so layers never meet and a relocated project keeps its payloads. - Stage each generation in a unique private directory and publish it by rename. A failure removes only the staging directory, identical content reuses the published generation (g-<content hash>), and changed content is published beside it before the superseded one is removed. Stale staging left by an interrupted process is swept after an hour. - Materialize cache hits as c-<entry key>, reusing an intact copy without writing and replacing a damaged one. - Drop the ownership record, adoption, byte comparison, and in-place replacement; derive payload names in one place and append manifest entries only after the generation is published. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the "reduce temporary generated-layer ownership" direction from the roadmap and ecosystem strategy, which also asks that ownership, crash recovery, and test isolation of generated files be explicit. The second commit reworks the first commit's design after code review; see below.
tile=truelayer a second time (new process, or after the first layer was released) failed withLAS016/COPC007because the earlier payloads already existed.Could not open assetin tiled COPC reads.UsdStage::LoadNone, so a read never resolves the payloads it is writing.ScopedLayerIdentifier/.usdgeo-stream-Nidentifier rewriting is removed.<payloadDirectory>/<owner>/<generation>/.<owner>is a hash of the source (for local files, its path relative to the payload directory, both canonicalized) plus the layer's exact file-format arguments. Layers never share or touch each other's files, and a project that is moved or mounted elsewhere keeps its payloads..tmp-<token>staging directory and published by rename. A failed or cancelled read removes only its staging directory, and a published generation is never modified.g-<content hash>generation. Changed content is published beside it, and the root points at the new paths, so layers still registered from the old generation never become stale. The superseded generation is removed on a best-effort basis.c-<entry key>. An intact copy is reused without writing, so a published directory may be read-only; a damaged copy is replaced. No byte comparisons and no adoption.AuthorPointCloudTiledAssetWithPayloadsoverload that returns typed diagnostics.uintmax_tanduint64_tdiffer. CI does not build benchmarks.docs/architecture/FILE_FORMAT_ARGUMENTS.md, "Generated Payload Ownership". Capability matrix, implementation status, streaming roadmap, OpenUSD surface, adapter and workspace contracts, module READMEs, and CHANGELOG are updated to match.Review follow-up (second commit)
The first commit recorded ownership per payload name in a shared flat directory. Review found that this let:
.tmpnames;std::to_string-rounded arguments collapse distinct layers onto one owner;Per-owner, immutable, rename-published generations remove those cases structurally, together with the record file, the adoption rule, and the in-place replacement.
Compatibility
Payload files that earlier tiled reads wrote directly into
payloadDirectoryare no longer used or touched. Delete the oldTile_*.usdcfiles to reclaim the space.Follow-up
Tracked as an open item in implementation status: tile spools still live in timestamped system-temp directories that an interrupted process leaves behind.
Validation (macOS arm64, cy2026/usd)
ost build+ost test: 18/18USDGEO_BUILD_BENCHMARKS=ON: 25/25c-<entry>copy is reused without writing, replaced when damaged, and user files are untouchedusdcat(LAS, COPC):g-<hash>with identical arcs (usdc output is deterministic)git diff --checkWindows and Linux are left to CI.
🤖 Generated with Claude Code