feat: Adding hand-crafted precompute engine option in e2e quickstart

[zzylol]

Summary

• Replaces the Arroyo streaming pipeline (Kafka + Arroyo + asap-summary-ingest) with the hand-crafted precompute engine running inside the query engine process
• Simplifies the e2e quickstart from ~12 services down to ~9, removing Kafka, Arroyo, kafka-init, and asap-summary-ingest
• Adds Precompute variant to StreamingEngine enum and makes Kafka CLI args optional
• Cross-group watermark propagation: two-layer watermark design (intra-worker max, cross-worker min via shared atomics) so idle groups get their windows closed
• Migrates SimpleMapStore from legacy to current (epoch-based) store implementations
• E2E resource test: standalone binary that simulates 189K series and reports CPU/memory

Data flow change

Before (Arroyo): Fake Exporters → Prometheus → Arroyo (via remote_write) → Kafka → asap-summary-ingest → Query Engine Store

After (Precompute): Fake Exporters → Prometheus → Query Engine precompute ingest (via remote_write) → Store (direct, zero-copy)

Code changes

Precompute engine

File	Change
precompute_engine/worker.rs	Refactor from per-series to per-group accumulators (GROUP BY semantics); add cross-group watermark propagation via shared Arc<AtomicI64>; fix panic on unknown agg_id (return Option)
precompute_engine/engine.rs	Create shared watermark atomics per worker; pass to Worker::new; add worker_watermarks to PrecomputeWorkerDiagnostics
precompute_engine/series_router.rs	Route by hash(agg_id, group_key) instead of series key; add GroupSamples message variant
precompute_engine/ingest_handler.rs	Group decoded samples by (agg_id, group_key) before routing; match series to aggregation configs
precompute_engine/precompute_engine_design_doc.md	Add watermark figure (event time, window lifecycle, late data) and cross-group propagation design section

Query engine integration

File	Change
data_model/enums.rs	Add Precompute to StreamingEngine enum
main.rs	Make --kafka-topic/--input-format optional; skip Kafka consumer when --streaming-engine=precompute; auto-enable precompute engine; add memory diagnostics logging
bin/e2e_quickstart_resource_test.rs	New: standalone E2E test simulating 7 exporters x 27K series for resource usage reporting

Store migration

File	Change
stores/simple_map_store/mod.rs	Switch SimpleMapStore enum to wrap SimpleMapStoreGlobal/SimpleMapStorePerKey (current) instead of legacy; move StoreDiagnostics/AggregationDiagnostic structs here
stores/simple_map_store/global.rs	Add diagnostic_info() method
stores/simple_map_store/per_key.rs	Add diagnostic_info() method

Bug fixes

File	Change
asap_types/streaming_config.rs	Fix panic on invalid aggregationId -> proper anyhow error
precompute_engine/worker.rs	Fix flush_all no-op (was calling closed_windows(wm, wm) which always returns empty)

Docker / quickstart

File	Change
docker-compose-precompute.yml	New: base compose with Prometheus, query engine, 8 fake exporters (adds missing fake-exporter-spiky)
docker-compose-precompute.dev.yml	New: override to build query engine from source
docker-compose-precompute.local.yml	New: override to use pre-built local binary
Dockerfile.queryengine-local	New: lightweight image copying pre-built binary
config/prometheus-precompute.yml	New: Prometheus config pointing remote_write to query engine
.dockerignore	Add target/ and .git/
.gitignore	Add asap-quickstart/bin/

Quickstart benchmark results

Resource usage: Query engine 39% CPU, 30 MiB memory (vs Prometheus 37% CPU, 654 MiB)

Quantile query latency (ASAP vs Prometheus, 189K series):

Query	ASAP	Prometheus	Speedup
p99	11ms	622ms	57x
p95	12ms	693ms	58x
p90	12ms	583ms	49x
p50	16ms	635ms	40x

Accuracy: <1% error at p95/p99, <7% at p50 (expected for KLL K=200)

Test plan

• cargo check passes
• cargo test -- all 434 tests pass (23 worker tests including 5 new watermark tests)
• cargo fmt --check passes
• Docker quickstart runs end-to-end with correct quantile results
• Existing --streaming-engine=arroyo mode unaffected (backward compatible)