Skip to content

fix(benchmarks): snapshot sentinel + earlier lineage assert in T0 merge driver - #3157

Merged
Xore merged 3 commits into
mainfrom
issue-3139
Sep 9, 2026
Merged

Xore merged 3 commits into
mainfrom
issue-3139

Conversation

@Xore

@Xore Xore commented Sep 9, 2026

Copy link
Copy Markdown
Owner

Summary

Two non-blocking follow-ups from the #3137/#3138 review (F3/F5), both in round7_t0_merge_export.sh:

  • Snapshot completion sentinel. The base snapshot_download stage had no completion marker — have_base_snapshot() only checked for model.safetensors.index.json or model.safetensors, either of which a partial download can leave on disk. A re-run then skipped the download and failed later at convert. Added .snapshot_done, mirroring the existing .merge_done sentinel in the merge stage: write-on-success, check-before-skip, and rm -rf the snapshot dir on re-entry when the marker is absent so a stale partial download can't linger.
  • Manifest lineage assert moved earlier. base_f16_sha != merged_f16_sha used to run in the manifest stage (stage 5), after ollama create had already registered all four tags. Moved to a new stage 3c, right after both f16 GGUFs exist and before stage 4 registers anything — a lineage collision now aborts before any tag is created instead of after. The manifest stage reuses the two shas computed there instead of re-hashing ~15GB twice.

No behavior change beyond failing earlier / more correctly on the partial-download edge case; T0_TAGS, quantize logic, and everything else untouched.

Note: the ops copy at /mnt-1/benchmarks/round7_t0_merge_export.sh on the homeserver will be re-synced from the blob after merge (orchestrator handles that).

Closes #3139

Test plan

  • bash -n round7_t0_merge_export.sh
  • shellcheck round7_t0_merge_export.sh clean
  • Manifest heredoc parsed with ast.parse after the edit
  • Full pipeline run on homeserver (gated on cold-run completion / RAM headroom, not exercised here)

round7_t0_merge_export.sh follow-ups from the #3137/#3138 review (F3/F5):

- Base snapshot_download stage now writes .snapshot_done the same way the
  merge stage writes .merge_done, and wipes the dir on re-entry when the
  marker is absent. Before, a partial download could leave index.json (or a
  lone shard) on disk, which have_base_snapshot() read as finished -- a
  re-run would skip the download and fail later at convert.
- The base_f16_sha != merged_f16_sha lineage assert now runs right after
  both f16 GGUFs exist (stage 3c), before stage 4 registers any tag in
  Ollama, instead of in the manifest stage after registration already
  happened. The manifest stage reuses the two precomputed shas instead of
  re-hashing ~15GB twice.

Closes #3139
@strix-security

strix-security Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

Strix Security Review

No security issues found.

Updated for 409e6bf.


Reviewed by Strix
Re-run review · Configure security review settings

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@strix-security strix-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the single changed file, analysis/ghidra/benchmarks/corpus/round7_t0_merge_export.sh (a privileged, operator-run benchmark orchestration script on a private homeserver). The PR adds a snapshot-completion sentinel (including a rm -rf of a config-derived, bounded subdirectory), moves a sha256 lineage assertion earlier in the pipeline, and refactors the manifest generator to reuse precomputed hashes passed as argv. All interpolated variables are quoted, operator-configured with safe defaults, or derived from local trusted files (sha256 hex digests and stat sizes); none are attacker-controlled. No injection, authorization, secret, deserialization, SSRF, path-traversal, or other security-relevant issue was found in the changed lines. The change is a correctness/robustness refactor with no security impact.


Reviewed by Strix
Configure security review settings

…se fetch, hash guards

Round-7 T0 merge/export script review fixes (#3139):
- base snapshot sentinel moves from $BASE16 (root-owned, unsloth
  container writes it) to $RUN -- host-side `date > sentinel` was
  dying on every run
- drop unconditional rm -rf of $BASE16: it silently no-ops
  unprivileged against the root-owned tree, and discards HF cache
  resume metadata forcing a full ~15G re-download; snapshot_download
  resumes on its own, sentinel alone was the actual bug
- guard base GGUF f16 convert output with -s check, matching the
  merge stage's have_weights guard
- guard lineage-assert shas non-empty before comparing, so a
  sha256sum failure under no-pipefail can't silently short-circuit
  to a false negative

@strix-security strix-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the single changed file, analysis/ghidra/benchmarks/corpus/round7_t0_merge_export.sh. The changes add a .base_snapshot_done completion sentinel around the Hugging Face snapshot download and move the base-vs-merged f16 lineage assertion to a new stage 3c, reusing the computed SHA-256 digests in the manifest stage instead of re-hashing. All new shell variables are quoted, no eval or command-substitution-to-shell sinks are introduced, no secrets are added, and there is no attacker-controlled input — the script is an operator-run local benchmark pipeline with fixed model paths, revisions, and integrity hashes. Static analysis (semgrep security/secret rules) reported no findings and bash -n passed. No security issues found in the changed code.


Reviewed by Strix
Configure security review settings

@strix-security strix-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the single changed file, analysis/ghidra/benchmarks/corpus/round7_t0_merge_export.sh. The PR adds a .base_snapshot_done completion sentinel to guard against partial-download re-entry, and moves the base-vs-merged f16 SHA lineage assertion to an earlier stage (3c), reusing those two hashes in the Python manifest step instead of re-hashing ~15GB files. The added shell commands (sha256sum, date, [ ... ]) are correctly quoted against fixed internal paths, and the two SHA values are passed to the Python heredoc as separate argv elements used only as plain strings. No injection, hardcoded secrets, path traversal, SSRF, or other security-relevant patterns are introduced by the diff. No security issues found.


Reviewed by Strix
Configure security review settings

@Xore
Xore merged commit 6573f3f into main Sep 9, 2026
192 of 206 checks passed
@Xore
Xore deleted the issue-3139 branch September 9, 2026 22:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

benchmark: t0 merge driver follow-ups — snapshot completion sentinel, early manifest-lineage assert

1 participant