Skip to content

Publish cache copies atomically so Plex never sees a partial file - #208

Merged
Brandon-Haney merged 1 commit into
StudioNirin:mainfrom
Brandon-Haney:pr/atomic-cache-publish
Sep 3, 2026
Merged

Publish cache copies atomically so Plex never sees a partial file#208
Brandon-Haney merged 1 commit into
StudioNirin:mainfrom
Brandon-Haney:pr/atomic-cache-publish

Conversation

@Brandon-Haney

Copy link
Copy Markdown
Collaborator

Fixes #207.

What happens today

On Unraid the same share-relative path can exist on both the pool and the array, and shfs answers /mnt/user/<share>/... with the pool copy. PlexCache copies media straight to its final name on the pool, so that name exists at zero bytes the moment the copy opens it, and stays partial for the whole transfer. Plex reads the share through that same union, so it can read the file mid-copy.

What Plex does with it

Measured on Unraid 7.3.1 with Plex 1.42, using a probe file deliberately shadowed on the pool. Run twice, same result both times:

Pool copy What Plex recorded
whole correct size, item healthy
truncated ingested the truncated size
zero bytes item removed from the library
restored to whole still gone, a scan brought it back under a new ratingKey

The zero byte row is the one that bites. Every copy is zero bytes for a moment at the start, and if Plex reads in that window the item drops out of the library and does not return on its own.

The change

Copy to a dot-prefixed .pc-part name in the same directory, verify the size against the source, then rename into place. A same-directory rename is atomic, so the real name goes from absent to complete with no observable middle state. The dot prefix means Plex's scanner ignores the temporary file if it surfaces through the share.

Failure cleanup now removes the partial name as well as the real one. A crash or an OOM kill skips cleanup entirely, so cleanup_stale_partials() sweeps the destination folders before each cache run and removes any .pc-part older than an hour. The age floor keeps it away from a copy that is still in flight.

How to test

  1. Queue a cache operation that moves a reasonably large file.
  2. While it is copying, list the destination folder on the pool. Only .Movie.mkv.pc-part should be there. The real filename should not exist yet.
  3. When the copy finishes, the real filename is present at full size and no .pc-part remains.
  4. Confirm the Plex library item stays healthy through the whole transfer, including its file size.
  5. Interrupt a run mid-copy (stop the container during a copy). The .pc-part stays behind. On the next cache run to that folder, the log shows [CACHE] Removed stale partial copy from an interrupted run.

Automated coverage in tests/test_atomic_cache_publish.py: the real name never exists while copying, a short copy is rejected before publishing, the partial is removed when the copy raises, a stale partial does not block a retry, and the sweep leaves recent partials and real media alone. Three existing FUSE path safety tests were updated so their stubbed copy registers its destination, since the destination is now the partial name.

Verified on a live run.

Unraid's shfs answers /mnt/user/<share>/... with the pool copy whenever
the same share-relative path exists on both the pool and the array. Media
was copied straight to its final name on the pool, so that name existed at
zero bytes the moment the copy opened it, and stayed partial for the whole
transfer. Plex reads the share through that same union.

Measured on Unraid 7.3.1 / Plex 1.42 with a shadowed probe, twice:

  pool state        what Plex recorded
  ----------------  ---------------------------------------------
  whole             correct size, item healthy
  truncated         ingested the truncated size
  zero bytes        item removed from the library
  restored          still gone; a scan brought it back as a new
                    ratingKey

Copy to a dot-prefixed .pc-part name, verify size against the source,
then rename into place. The rename is same-directory and therefore
atomic, so the real name goes from absent to complete with no observable
middle state. Refs StudioNirin#207.

Also sweeps .pc-part files left by an interrupted run, since a crash
skips _cleanup_failed_cache_copy and nothing else references them.
@Brandon-Haney
Brandon-Haney merged commit 2a584cb into StudioNirin:main Sep 3, 2026
2 checks passed
@Brandon-Haney
Brandon-Haney deleted the pr/atomic-cache-publish branch September 3, 2026 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Plex loses files sometimes (Not actually PlexCache-D fault but caused by it)

2 participants