feat(scripts): M1 end-to-end smoke (Deribit → ingestion → engine → API → WS)

[obchain]

Summary

M1 close gate per issue #66. scripts/e2e-smoke.sh boots every M1 service in dependency order, polls for two engine snapshots so the pipeline is warm, then runs five hard assertions covering both the storage layer and the API surface.

After this lands, the published claim "the local pipeline works end-to-end" is enforceable by a single command.

What the script does

docker compose down -v + up -d     →   wipe stale data, start clickhouse + redis
healthcheck poll                    →   wait for `/ping` 200 + `redis-cli PING`
cargo build --release               →   pre-build ingestion + engine
go build ./cmd/api                  →   pre-build the API
spawn ingestion, engine, api        →   background processes with per-stage logs
poll /v1/health                     →   wait for API readiness
poll index_ticks for ≥ 2 snapshots  →   engine warm, two-cycle stable
assert 5 checks                     →   options_ticks, index_ticks, /latest, /history, WS
teardown via EXIT trap              →   reap rust + go binaries by name, compose down

Assertions

#	Check	Threshold
1	options_ticks fresh rows	≥ 1 in last 60 s
2	index_ticks fresh rows	≥ 1 in last 120 s
3	GET /v1/index/bvol/latest	200 + value > 0 + age < 90 s
4	GET /v1/index/bvol/history?interval=5m&limit=12	200 + bars ≥ 1
5	ws:///v1/stream subscribe	one tick per channel inside 75 s

scripts/e2e-ws-client.py is the WS verifier — strict frame-shape check against the PRD §6 contract (type:"tick", lowercase channel, integer ts ms, float value/confidence), exits non-zero with a diagnostic when a channel goes silent. Reused by the script and runnable standalone against an already-up pipeline.

Live verification (dev box)

==================================== summary ====================================
stage                       seconds
-------------------------- --------
compose-up                        3
compose-healthy                  11
build-rust                        1     (cache warm)
build-go                          1
api-ready                         2
engine-cycles                   120     (two 60 s cycles)
assert-options_ticks              0     fresh rows: 38188
assert-index_ticks                0     fresh rows: 4
assert-rest-latest                0     bvol=37.23 age=1s
assert-rest-history               0     bars=1
assert-ws-stream                 60     bvol + evol both delivered
-------------------------- --------
TOTAL                           198
=================================================================================
OK

WS frames received in PRD wire shape, exact:

[ws] tick bvol value=37.1958 ts=1779803300902 confidence=1
[ws] tick evol value=49.9745 ts=1779803300902 confidence=1

Robustness fixes baked in

Each of these surfaced during dev-box verification:

Symptom	Fix
declare -A: invalid option on stock macOS	Replaced associative arrays with parallel indexed arrays so the script runs on bash 3.2 (the macOS default)
Tried to spawn volx-normalizer	The crate is a library embedded in the ingestion process, not a binary. Removed from service list
Syntax error: failed at position 1 ('query') from ClickHouse	curl --data-urlencode POSTs the form-encoded body; needs -G to send as URL param. Helper now uses -G
Stale index_ticks from a previous session satisfied "fresh row" assertions	Compose down --volumes at script start guarantees a clean slate
/latest age (107s) ≥ 90s failure on cold runs	Wait for ≥ 2 distinct snapshots (engine warm) before the latest-age check, instead of just 1
cargo run cold-compile blew the 60 s api-ready timeout	Explicit build phase ahead of the service-start stage; runtime binaries are launched directly
cargo run fork-exec child not reaped on teardown	EXIT trap kills by binary name (target/release/volx-ingestion, etc.) plus the parent PID

File map

File	Role
scripts/e2e-smoke.sh	Orchestrator, 200 lines bash 3.2-compatible
scripts/e2e-ws-client.py	WS frame-shape verifier (50 lines, websockets lib)
README.md "End-to-end smoke" section	One-line invocation + dependencies + per-assertion table
.gitignore	Gitignore api/api-bin produced by the build phase

Dependencies

Tool	Purpose
docker (compose v2)	clickhouse + redis containers
cargo (Rust ≥ 1.85)	ingestion + engine build
go (1.25)	api build
curl	health probes + ClickHouse HTTP queries
python3 with websockets	the WS verifier (script auto-discovers the M0 research venv)

Configuration

Env var	Default	Notes
ENGINE_WAIT_S	135	poll deadline for ≥ 2 snapshots
WS_TIMEOUT_S	75	WS verifier per-channel deadline
CLICKHOUSE_DB	volx	matches docker-compose default
PYTHON_BIN	auto-detect	prefers research/.venv/bin/python3

Out of scope

• Wiring the script into GitHub Actions CI — that's issue GitHub Actions: CI (lint + test, no deploy) #28, which calls this script as the integration job
• Multi-venue assertions — M2 ingestion-okx: OKX options WebSocket connector #59 / ingestion-bybit: Bybit options WebSocket connector #60 will extend the script with WHERE venue IN (...) checks once OKX + Bybit ingest land
• Headless-browser assertion (Playwright / Puppeteer) — the Python WS client proxies the browser's tick contract; a real browser-render check is a separate test-tooling issue
• Performance thresholds on the stage-timing table — the table is observational v1; SLOs are M3+

Test plan

• bash -n scripts/e2e-smoke.sh clean on macOS bash 3.2
• python3 -m py_compile scripts/e2e-ws-client.py clean
• Full live smoke on the dev box: all 5 assertions pass, 198 s total
• Re-run immediately after the first run — second run also passes (idempotency via down --volumes + binary-name pkill)
• Manually killed engine mid-run on a separate execution to confirm the /latest age assertion fails with the expected message

Fixes #66