Phase B5: sensor-trust scoring + adaptive Kalman by bbox area - #13
Merged
Merged
Conversation
Phase A1 of the 10x roadmap. The README advertised several capabilities as shipped that aren't wired into the live pipeline; this marks them as planned and fixes concrete drift. - Add an "Implementation status" table up front (source of truth: ✅ vs 🔭) - Mark 🔭 planned: cross-camera homography (H-PROJ), pixel extrapolation (EXTRAP), appearance re-ID, sensor-trust scoring, adaptive-Kalman-by-area, GPS/IMU fusion, DeepSORT/centroid fallback chain, compass ribbon, threat ring - Reframe the homography section as forward-looking design - Note only the orange WORLD ghost path is emitted today - Fix mobile handshake (no target_fps is sent; sensor_data is received, not fused) - Update Testing section for the new frontend tests + comprehensive CI gate - pyproject: allow Python 3.13 (requires-python <3.14; verified locally) - .env.example: remove the duplicated JWT/AUTH block No code behaviour change; backend gate green (ruff/mypy/57 tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…AMERA_POSITIONS Phase A2 of the 10x roadmap. Previously, with no CAMERA_POSITIONS configured, pixel_to_world() returned None for every camera, so the world model produced ZERO world objects and ZERO predictions — viewers saw only raw detections and per-camera tracks, and the "fusion" layer silently no-opped. - Add WorldModelRepositoryImpl._ensure_calibration(): lazily synthesize a default calibration (cameras spread along x-axis, height 2.5 m, focal 800) when a camera has none, with a one-time warning. Explicit CAMERA_POSITIONS still take precedence. - Wire it into _process_track and generate_predictions so both track fusion and view-only prediction cameras work out of the box. - Tests: 4 new (objects created without config; lazy creation for unseen camera; explicit positions win; predictions don't raise for a second camera). - Docs: backend/ARCHITECTURE.md no longer claims CAMERA_POSITIONS is required. Backend gate green: ruff + mypy clean, 61 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase A3 of the 10x roadmap. CameraTracker.update() previously mapped Hungarian row indices back to tracks via `list(self.tracks.keys())[idx]` inside its loops — O(n^2) and implicitly coupled to dict iteration order matching the cost-matrix build order. Correct today, but fragile. - _associate_detections now returns (matched[(track_id, det_idx)], unmatched_track_ids, unmatched_det_indices); update() consumes track ids directly. Behaviour-preserving; removes the ordering coupling and the per-match key-list rebuilds. - Add 6 characterization tests for the CameraTracker lifecycle (create → confirm → coast → remove), including multi-track id-mapping correctness. Coverage: tracking_adapter 46%→85%, total 42%→49%. ruff/mypy clean, 67 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase A4 of the 10x roadmap — completes Phase A (the honest & solid base). - Add msgpack round-trip tests for WebSocketCommunicationRepository._serialize_* (snapshot envelope, JPEG frame encoding, detection/track/world-object/prediction shapes, empty snapshot). This pins the exact wire format the frontend decodes. - CI: install fastapi + msgpack (needed to import the ws adapter) and raise the coverage floor 40% -> 50%. websocket_adapter 0%->46%, total 49%->52%. ruff/mypy clean, 74 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… matching Phase B1 — the first feature build. Activates appearance re-ID, which was scaffolded but dormant (compute_appearance was never called; Detection.appearance was always None, so tracking was pure-IoU and cross-camera matching distance-only). - detection: extract module-level compute_hsv_appearance() and populate Detection.appearance per detection (64-dim L2-normalized HSV histogram; toggle via APPEARANCE_REID_ENABLED). This also activates the appearance term already present in the tracking cost matrix. - world model: gate cross-camera association on appearance cosine similarity (CROSS_CAMERA_APPEARANCE_THRESHOLD, default 0.5) so differently-dressed people at the same ground position stay separate; EMA-smooth (alpha=0.3) the fused descriptor. - config: add appearance_reid_enabled + cross_camera_appearance_threshold as real Settings fields (env-wired) + .env.example docs. - README: flip "Appearance re-ID" from planned to implemented. - Tests: 7 new (descriptor shape/similarity/degenerate; appearance-gated matching; distance fallback; config wiring). Coverage 52%->55%. ruff/mypy clean, 81 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase B2. Implements the headline cross-camera homography feature: when one camera
sees a person and another doesn't, project the foot point between views to render a
green H-PROJ ghost.
- New app/infrastructure/homography.py: HomographyEstimator accumulates foot-point
correspondences per camera pair, estimates H via cv2.findHomography + RANSAC once
>= min_pairs, re-estimates periodically, projects points, caps the pair buffer.
- World model integration:
- store per-camera foot points on WorldObject (camera_foot_points)
- collect correspondences from objects co-visible on a tick (_collect_correspondences)
- generate_predictions tries Path A (homography, HOMOGRAPHY method) first, falling
back to Path C (world projection) when no homography exists
- wired from the existing HOMOGRAPHY_* Settings
- README: flip homography / H-PROJ from planned to implemented.
- Tests: 8 new (estimator: known-H recovery, identity, directionality, min-pairs,
buffer cap, same-cam ignore; integration: predictions emit HOMOGRAPHY, correspondence
collection for co-visible objects).
Coverage 55%->58%. ruff/mypy clean, 89 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase B3 — the third and final ghost-prediction path. When a camera loses a person it was previously tracking (but the world object is still alive via another camera), dead-reckon a red EXTRAP ghost from this camera's last-known pixel position along its pixel velocity, capped by an adaptive budget. - world model: refine the prediction skip from "ever seen by this camera" to "seen within a live window" (~2 frames), so a camera that LOST a target becomes eligible for a ghost. generate_predictions now tries Path A (homography) -> Path B (extrapolation) -> Path C (world projection). - _try_extrapolation_prediction: slide last pixel by velocity * time * fps, capped at min(250, 80 + 40*t) px; zero-velocity stays put; no pixel history -> None. - README: flip EXTRAP / red ghost from planned to implemented; all 3 paths now active. - Tests: 6 new (no-history None, moves in velocity direction, budget cap, zero-velocity stays, integration emits EXTRAP for a lost camera, live camera skipped). Coverage 58%. ruff/mypy clean, 95 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase B4. The mobile client streams GPS (watchPosition) + IMU (DeviceOrientationEvent), but the backend was receiving and discarding the sensor_data message. Now it's fused. - New app/infrastructure/geo.py: gps_to_local() equirectangular projection. - World model: update_camera_sensor() converts GPS -> local position (relative to GPS_REFERENCE_* or the first fix as origin) and DeviceOrientation alpha/beta/gamma -> camera rotation, overriding the auto-default calibration. Added to the WorldModelRepository port (concrete no-op default). - main.py: /ws/camera handler forwards sensor_data to the world model instead of pass. - README: flip GPS/IMU fusion from planned to implemented. - Tests: 8 new (geo: origin/north/east projection; fusion: first-fix origin, north offset, configured reference, orientation->rotation, no-data no-op). Coverage 58%->59%. ruff/mypy clean, 103 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase B5 — the final roadmap feature. The Kalman update accepted a sensor_trust param but it was always 1.0, and measurement noise scaled only by confidence. - KalmanFilter.update now scales R by confidence * sensor_trust * bbox-area factor, and returns the innovation magnitude. - World model tracks per-camera trust in [0.1, 1.0]: consistent measurements (low innovation) raise it, outliers decay it; _bbox_area_factor maps bbox area to a [0.1, 1.0] reliability factor (larger = closer = lower noise). Both feed kf.update. - config: sensor_trust_innovation_threshold + bbox_reference_area Settings (env-wired) + .env.example docs. - README: flip sensor-trust + adaptive-Kalman-by-area to implemented; refresh the status note (remaining planned items are the deferred DeepSORT/compass/threat ring). - Tests: 8 new (innovation return, area-factor effect, trust up/down/clamped, default, area-factor bounds, config wiring). Coverage 59%->60%. ruff/mypy clean, 111 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jun 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Phase B5 — the final feature of the 10× roadmap. The Kalman
updateaccepted asensor_trustparam but it was always1.0, and measurement noise scaled only by confidence.Changes
KalmanFilter.updatenow scales measurement noiseRbyconfidence × sensor_trust × bbox-area factor, and returns the innovation magnitude.[0.1, 1.0]: consistent measurements (low innovation) raise it, outliers decay it._bbox_area_factormaps bbox area → a[0.1, 1.0]reliability factor (larger bbox = closer = lower noise). Both feedkf.update. Exposed viaget_sensor_trust.sensor_trust_innovation_threshold+bbox_reference_areaas realSettingsfields (env-wired) +.env.exampledocs.Tests (TDD)
8 new in
test_sensor_trust.py: innovation return value, higher area-factor trusts the measurement more, trust rises on consistent / decays on outlier / clamped to[0.1,1], default trust = 1.0, area-factor bounds, config wiring.Verification
ruffclean ·mypyclean (18 files) · 111 tests pass (was 103).🎉 Roadmap complete
With this PR, every feature the README advertises is genuinely implemented and tested. The only items still marked 🔭 are explicitly deferred (DeepSORT fallback chain) or out of scope (compass ribbon, threat ring).
🤖 Generated with Claude Code