Repository navigation
Conversation
…erlay patch the bundle recorded A bundle is never re-overlaid, so `fluid validate <bundle>.tgz` could not see an overlay patch that landed on another entry (overlay_patch_retargeted): only `fluid bundle --env` logged it, and pipeline stage 2, which runs exactly `fluid validate <bundle> --env <env> --strict`, passed. `fluid bundle` now merges the overlay inside collect_overlay_warnings(), says each warning once, and records the messages in MANIFEST.json as source.overlayWarnings, only when there are any. The source block is outside the merkle root, so the digest still names the contract alone. Validate reports each one as a warning finding (overlay, MANIFEST.json, OVERLAY-WARNING), so --strict fails on it, and --format json and --report list it. A bundle that records none, one built before this change included, validates as before. The bundle path now prints what the contract path prints. A passing bundle lists its warnings, not only their count. --quiet, which printed nothing, prints ERROR: lines, a WARNING: line for a recorded overlay warning, and under --strict an ERROR (--strict): line for each warning. When warnings alone fail --strict, it says so. Borrowed: the split SLSA Provenance v1 makes between the artifact digest (subject) and run metadata (runDetails). The MANIFEST source block already has that shape. Not SARIF, which is a full diagnostics format, for one list of strings in an advisory block.
…s what it took The caller reports the warnings a block took only when the block completes, and the loader logs them at DEBUG only. So when another overlay entry failed the merge after a patch had been retargeted, the retargeted patch was never shown. `fluid bundle --env` printed the merge error alone, where main printed both, and `fluid validate --env` and `fluid contract-validation --env` lost it the same way. The block now logs what it took, as the loader logs it, before the error propagates.
…terminal The record is outside the merkle root, so editing it keeps the bundle digest. An entry carrying escape sequences could move the cursor, erase or forge lines, or write the clipboard in a permissive terminal. A lone surrogate made validate crash before it wrote --report. Each character that is not printable is now escaped as repr writes it, as the block's other values are quoted with !r. _cprint_literal also passes highlight=False, as _cprint_json does. Rich's highlighter takes time quadratic in a run of `<` (4 s for 10,000), and the record put such a run on a passing bundle's output.
📄 Documentation ReminderThis PR appears to be missing a documentation reference. Our docs live in a separate repo. Please update the PR description with one of:
See the Contributing Guide for details. |
…test.py leaks tests/cli/conftest.py sets FLUID_LOG_LEVEL=DEBUG, and some xdist workers keep it. The CLI subprocesses here inherited it, so the loader's DEBUG copy of a warning the command reports was printed too. The tests that count the warning once then failed with `assert 2 == 1`, as #734's TestValidateStrict did on main's CI run for 1320619. The subprocesses now drop FLUID_LOG_LEVEL, FLUID_DEBUG and FLUID_SHOW_REGISTRATION, as tests/output_ports already do, with API keys as before.
CodeQL read the assertion after `with pytest.raises(...)` as unreachable, because the block's last statement raises. The block now calls a helper that merges and then raises, and the test asserts the error message too. It still checks that a warning taken inside the failing block is logged.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #734, which made
fluid validate --env <env> --strictfail on an overlay patch that lands on another entry (overlay_patch_retargeted) but left the bundle path out. Pipeline stage 2 runs exactlyfluid validate <bundle> --env <env> --strict, and a bundle is never re-overlaid, so validate on the frozen bundle could not see the warning: it was only logged byfluid bundle, and the bundle passed.What changes
fluid bundle --envmerges the overlay insideloader.collect_overlay_warnings(), says each collected warning once, and records it in MANIFEST.json undersource.overlayWarnings. The key is written only when there is a warning. Thesourceblock is outside the merkle root, so the bundle digest is unchanged (the sample bundle keeps main'ssha256:6238f9…). For--format tgzit printsoverlay warnings: N recorded in the bundle; ....fluid validate <bundle>.tgzreports each recorded warning as a finding ([warning] overlay: MANIFEST.json: ..., codeOVERLAY-WARNING), so--strictfails on it. It shows in--format jsonand--reporttoo.--quietused to print nothing at all; it now printsERROR:lines, a recorded overlay warning asWARNING:, and under--stricteach warning asERROR (--strict):. When warnings alone fail--strict, it saysValidation failed: warnings treated as errors (--strict).Two fixes the pre-PR defect scan found, as separate commits:
fluid bundle, and it was already the case in feat: trigger.type continuous (0.7.6 preview) is never scheduled; --strict fails on a retargeted overlay patch #734'sfluid validate --envandfluid contract-validation --env. The fix is incollect_overlay_warnings, so it covers all three.overlayWarningskeeps the digest. Escape sequences in it could move the cursor or forge lines, and a lone surrogate crashed validate before it wrote--report. Non-printable characters are now escaped asreprwrites them, as the block's other values are quoted with!r._cprint_literalalso passeshighlight=False, like_cprint_json, because Rich's highlighter is quadratic in a run of<.Not in this PR
fluid contract-validation <bundle>andfluid test <bundle>do not read the record. The CHANGELOG says so and points at the contract file with--env. Generated pipelines run those commands on the contract file.expecteddigests in MANIFEST-TAMPER. That is already the case on main.Documentation
forge_docs#158 updates
cli/validate.md(the bundle bullet under--env, "Validating a bundle", andOVERLAY-WARNING),cli/bundle.md(source.overlayWarnings) andconcepts/environments-and-overlays.md. It needs review.Also fixed: the overlay CLI tests under a leaked debug level
tests/cli/conftest.pysetsFLUID_LOG_LEVEL=DEBUG, and some xdist workers keep it. At DEBUG the loader also logs a warning the command reports, so #734'sTestValidateStricttests counted it twice (assert 2 == 1). That is how they failed on main's CI for1320619b. The CLI subprocesses intests/test_overlay_keyed_merge.pynow dropFLUID_LOG_LEVEL,FLUID_DEBUGandFLUID_SHOW_REGISTRATION, astests/output_portsalready does. The file passes with and withoutFLUID_LOG_LEVEL=DEBUGset.Borrowed
subject) apart from run metadata (runDetails). The MANIFESTsourceblock already had that split, so the record is metadata about the build and not bundle content.Test plan
tests/test_overlay_keyed_merge.py:TestValidateBundleStrict(11 tests), the reader unit test and a collector test. Every test that can fail did fail against the code before its fix: 8 against main, and the 4 gate-fix tests against the first cut. The other 2 pass on main by design: they are additivity guards (a keyed overlay records nothing, and a bundle without the record still passes--strict).fluid bundle contract.fluid.yaml --env prod --format tgzwith a positional 0.7.5 overlay, thenfluid validate runtime/bundle.tgz --env prod --strict(the stage-2 command), exits 1 and names the patch.--quiet,--quiet --strict,--format json,--report, an env mismatch next to the warning,--out -YAML (stdout stays clean), hand-editedoverlayWarningsshapes, andfluid plan+fluid apply --dry-runon a bundle that carries the record.ruff check fluid_build/ tests/,black --check(24.10.0),lint-imports(6 kept) andscripts/mypy_strict.pyall pass.-n 6, ~50 min at load avg ~124): 27,059 passed. The 10 failures and 2 errors are all environmental or already on main:canonical/mainexport: the dlt allow-list tests (4) and the duckdb sandbox temp-root tests (2).bashlexis not installed in the local venv.sqlglotccompiled extension). For the run it was shadowed with a working pure-Python sqlglot through PYTHONPATH.