feat: detect and fail loudly on patch failures in setup/migration/solution scripts

[joellabes]

Problem: when switching to patch-based migrations, a patch could silently fail and cause downstream problems.

Change Summary

Harness exit code detection:

• Adds run_script_checked helper that captures shell script exit codes via a temp file, since session.send_keys(..., block=True) only waits for tmux completion and never sees the command exit code
• Wires exit code checking into all three script execution paths: setup.sh (→ SETUP_FAILED), migration.sh (→ SETUP_FAILED), solution.sh (→ UNKNOWN_AGENT_ERROR)
• Captures terminal pane output before returning on setup failure (so logs are always written)

Shell script strict mode pattern:

• Adds set -euo pipefail at the top of all setup.sh (65), migration.sh (13), and solution.sh (65) files so that an inner patch failure propagates to the script exit code
• Adds set +euo pipefail after the last patch/fi block in all scripts — patches run under strict mode, post-patch dbt commands (which may legitimately fail in puzzle-state tasks) run permissively
• Adds exit 0 at the end of all scripts so the final exit code is not inherited from a failing dbt run
• Removes dbt run || true escape hatches (now redundant — strict mode is already off when dbt runs)

CI validation for Snowflake patches:

• Adds scripts_python/validate_snowflake_patches.py — simulates the full Snowflake patch lifecycle for every task with a Snowflake variant: migration → setup patches → solution patches, each in an isolated temp directory
• Adds .github/workflows/validate-patches.yml — runs on PRs that touch migrations, shared projects, or task patches
• Coverage check ensures every .patch file in the repo is exercised (tasks with no Snowflake variant are excluded — covered by existing DuckDB CI)
• Pre-deletes files that solution patches will create, to simulate the real container state

Other:

• Consolidates analytics_engineering005 identical duckdb/snowflake setup patches into a single changes.patch
• Updates task templates to include set -euo pipefail / set +euo pipefail / exit 0 by default

Before: airbnb001 Snowflake/dbt-fusion migration silently dropped a hunk and the trial continued into the agent phase with a corrupted project state.

After: Any failing patch call exits the script non-zero, which the harness catches and records as SETUP_FAILED rather than continuing.

Known Issues (pre-existing, caught by new CI)

• airbnb003/setup/changes.snowflake.patch — hunks reference post-migration context lines that do not match; needs regeneration
• analytics_engineering007/solutions/changes.patch — STRPTIME context line broken by migration in fact_inventory.sql

Test Plan

• All 84 unit tests pass (uv run pytest tests/)
• Run airbnb001 DuckDB variant — setup should fail with SETUP_FAILED if patch does not apply cleanly
• Validate Snowflake patches CI passes (currently 2 pre-existing failures listed above)

🤖 Generated with Claude Code