Skip to content

Add manifest rebase command for cross-cluster source/dest remapping#6

Open
fluidnumerics-joe wants to merge 2 commits intomainfrom
feature/manifest-rebase
Open

Add manifest rebase command for cross-cluster source/dest remapping#6
fluidnumerics-joe wants to merge 2 commits intomainfrom
feature/manifest-rebase

Conversation

@fluidnumerics-joe
Copy link
Member

Summary

  • Adds xfer manifest rebase command to rewrite source_root, dest_root, source, and dest fields in an existing manifest using its already-stored relative path field
  • Either root can be omitted to keep it unchanged from the original manifest
  • Enables cross-cluster workflows where a manifest is built on one cluster (e.g. against a local VAST filesystem) and transfers are run on another cluster using a different endpoint (e.g. an S3 remote for the same data)

Test plan

  • Build a manifest against a local or S3 source
  • Run xfer manifest rebase --in manifest.jsonl --out manifest-rebased.jsonl --source-root s3remote:bucket/prefix and verify source_root/source fields are updated
  • Verify path (relative) field is unchanged in rebased output
  • Verify omitting --source-root or --dest-root preserves the original value
  • Shard and run transfers using the rebased manifest on a second cluster

🤖 Generated with Claude Code

fluidnumerics-joe and others added 2 commits March 11, 2026 14:16
Allows a manifest built on one cluster (e.g. against a local VAST path)
to be rebased to a different source or dest root before running transfers
on another cluster (e.g. via an S3 endpoint). The relative 'path' field
is preserved; source_root, dest_root, source, and dest are rewritten.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant