Camera-only BEV Occupancy with Temporal Stability and Closed-Loop Mining

Problem

Build a camera-only BEV occupancy model on nuScenes, supervised by LiDAR-derived BEV labels, with a focus on:

• temporal stability
• long-tail failure analysis
• systems discipline

Constraints

• Inference: camera-only (CAM_FRONT)
• Supervision: LiDAR-derived BEV occupancy + known mask
• Grid: 50m forward × ±25m lateral @ 0.5m resolution (100×100)
• Compute: CPU-feasible (mini splits)

Labeling

• Occupied: LiDAR points with z > 0.3m
• Free: LiDAR points with z ≤ 0.3m
• Known mask: occupied OR free evidence
• Honest limitation: low-height structures (curbs/islands) are ambiguously supervised

Models

• v1: single-frame CNN → BEV occupancy
• vT0: temporal baseline using 3-frame logit averaging
• v2: closed-loop retraining via hard-case oversampling (no architecture change)

Metrics

• Masked IoU / F1 (only where supervision exists)
• Flicker (flip-rate): temporal stability metric
• Slice metrics: distance bins + brightness proxy

Results (mini_val)

• v1: IoU 0.61 / F1 0.75
• vT0: similar IoU, ~35% lower flip-rate
• v2: IoU 0.63 / F1 0.77
• Hard-tail (top-40) loss ↓ 0.094, entropy ↓ 0.061

Key Findings

• Temporal aggregation improves stability, not accuracy
• Closed-loop mining reduces severity of worst cases
• Some failures persist due to label ambiguity, not model capacity

Failure Analysis

See reports/failure_taxonomy.md for detailed breakdown:

• raised curbs / traffic islands
• resurfaced roads
• thin vertical structures
• far-range sparsity
• occlusion boundaries

Reproducibility

• Fixed seeds
• Cached labels
• Regression tests for labels and determinism

How to Run

# check dataset
python scripts/check_nuscenes.py

# train v1
python -m src.train_v1

# evaluate
python -m src.eval_v1

# temporal stability
python -m src.eval_flicker

# closed-loop retrain
python -m src.train_v2