Drone Traffic Intelligence

Turning two raw drone clips of a Pune intersection into GPS-accurate vehicle trajectories, real-time speed and queue analytics, and network-level traffic reasoning — snapped onto the actual OpenStreetMap road network, not a hand-rolled approximation.

Every box is a live detection · every trail is that vehicle's full path since it entered frame · every ID is a tracked object, not a re-detection

---

What this actually is

Two DJI Matrice 3D hovering-drone clips of the same intersection (~6-7 minutes each, 4K, one near-nadir crossroads view and one wide oblique view) go in. What comes out, through five progressively harder levels:

1. Detect & track every road user — cars, motorcycles, auto-rickshaws, pedestrians, the works — with stable IDs across the whole clip.
2. Describe each one — colour, rough speed — where the footage actually supports it (and say clearly when it doesn't).
3. Aggregate into turning movements, an origin-destination matrix, modal split, speed profiles, and queue lengths.
4. Ground it in reality — the drone's own flight telemetry recovers each vehicle's real lat/lon, and every point gets snapped onto the real OSM road network.
5. Reason about the network — where congestion actually starts, inferred signal performance, where real driven paths diverge from the assumed road geometry, and a census of vehicles that look like they might be obstructing traffic.

Nothing here is hand-waved. Every level ships with its own written report (writeup/L1.md … writeup/L5.md) that includes what was tried, what broke, what got measured, and — just as importantly — what's still a known limitation rather than a hidden one.

See it work

	Detector tuned for the hard case. The oblique clip's distant highway lane collapsed to a handful of pixels per vehicle at default resolution — zero detections. Fixed by running inference at imgsz=2560 instead of tiling (SAHI was tried, then rejected: 67 minutes per clip and it double-counted objects across tile boundaries). This frame alone has 150+ detections the naive settings would have missed entirely.
	Where speed concentrates. Aggregated from real per-detection speed estimates across the whole clip, gridded spatially — not eyeballed.
	Every dot is a real GPS point. ~818,000 detections, ground-projected from the drone's own gimbal telemetry and camera model, plotted against the actual OpenStreetMap road graph for this intersection. Median snap-to-road distance: 6.1 metres — on a road network pulled independently of the footage.
	Where real driving disagrees with the map. The solid lines are the actual mean paths vehicles drove between two approaches; the dashed lines are what a straight-line assumption would predict. One turning movement swings 34.85m wider than the naive geometry — a real, measured desire line, not an assumption.
	Every box, its own speed. Same tracking pipeline, now with per-vehicle km/h computed from a pixel-to-metre scale derived from real car dimensions — not a guess.
	Road-aware filtering. A hand-drawn "this is road" mask (deliberately rough — the user drew it in a hurry) gets registered into the tracking pipeline's own coordinate frame and used to tag every detection as on-road or not, so a stray rooftop or sidewalk detection can't quietly pollute a traffic count.

The five levels

L1	Detection & Tracking — VisDrone-tuned YOLO (9-class taxonomy that keeps auto-rickshaws as their own category instead of collapsing them into "other"), custom ByteTrack loop operating in a stabilized coordinate frame so a hovering drone's micro-jitter doesn't masquerade as vehicle motion. 26,460 tracks · 9,140 frames (oblique) — 9,344 tracks · 11,971 frames (nadir), full-lifetime IDs with growing trail overlays.
L2	Object-Level Insight — colour via zero-shot CLIP classification, gated on resolution so a 40px blur doesn't get a confident-sounding wrong answer. Licence-plate OCR was scoped out entirely once a smoke test confirmed plates simply aren't legible at ~50m AGL — reported as a finding, not silently skipped. Rough kinematics from a car-footprint-derived pixel-to-metre scale.
L3	Aggregate Insight — turning-movement classification, an OD matrix + modal split, speed profiles, and queue-length estimates. Ships with an interactive HTML dashboard. Also where a real tracking-fragmentation bug was found (vehicles losing their ID mid-turn when briefly occluded), diagnosed, and partially patched — see below.
L4	Spatial Grounding — parses the drone's flight telemetry straight out of the video file's embedded subtitle stream, builds a camera model from the gimbal pose and lens FOV, projects every detection onto a ground plane, and snaps it onto the real OpenStreetMap road graph. Cross-validated by a Leaflet map the project's own reviewer checked by eye: "looks good with little drift."
L5	Network Reasoning — congestion tracing (one real, measured queue-onset event found: a 30m backup on one approach), inferred signal timing (no ground-truth signal data exists, so every number here is honestly labelled "inferred," not "measured"), desire-line analysis, and a static-obstruction census (35 + 41 candidates across both clips, explicitly candidates, not confirmed violations).

Built like an engineering log, not a demo

The thing that makes this project worth reading isn't that everything worked — it's the trail of things that didn't, and how each one got found and handled instead of hidden:

• SAHI tiling looked like the fix for far-field recall — until it cost 67 minutes a clip and silently double-labelled objects across tile boundaries. Dropped in favour of a single higher-resolution pass.
• A DNS routing bug in urllib3 meant the OpenStreetMap data pull worked from curl but failed outright from Python — one of two backend IPs for overpass-api.de refused connections, and urllib3's retry logic kept re-resolving back onto the dead one. Fixed by patching the one choke point every connection funnels through, including retries.
• A tracking-fragmentation bug quietly inflated "U-turn" counts to 89% of all movement classifications — vehicles losing their ID mid-intersection when briefly occluded by cross-traffic, then getting a fresh ID that fakes a same-approach return trip. Diagnosed by clustering entry/exit points and noticing one cluster sat suspiciously close to the intersection's centre. A same-vehicle re-stitching patch cut that down to 85% — real progress, honestly reported as partial, not claimed as fixed.
• Two separate speed-measurement bugs surfaced while building queue detection — one where a single frame of tracker jitter was enough to register a normally-moving vehicle as "stopped," another where a verified parked car showed instantaneous speeds up to 32.6 km/h from sub-pixel projection noise. Both fixed with contiguity and smoothing checks before a single queue-length number was trusted.

Every one of these is documented in the level's own writeup/*.md, not buried in commit history.

Repo structure

data/                    source clips + hand-drawn road-area masks
models/                  detector weights
scripts/                 one script per pipeline stage, L1 -> L5
outputs/
  L1_detection_tracking/ track logs, class-smoothed, overlay videos
  L2_object_insight/     colour attributes, kinematics
  L3_aggregate_insight*/ movement, OD, modal split, speed & queueing
  L4_spatial_grounding/  telemetry, camera model, ground points, road binding
  L5_network_reasoning/  congestion, signal inference, desire lines, obstructions
  road_mask/             on-road detection flags
  track_stitching/       the fragmentation patch fix's track-ID remap
writeup/                 one report per level + the L3 dashboard
docs/assets/             the images and clips in this README
planning.md              the full build log — decisions, dead ends, timings

Under the hood

ultralytics (YOLO + ByteTrack) · opencv-python · polars for million-row track logs without the wait · transformers/CLIP for zero-shot colour · osmnx + geopandas + shapely for real road-network binding · pyproj for CRS-correct ground projection · plain Leaflet/HTML for the maps, no map SDK lock-in.

Read more

• planning.md — the full decision log: what was tried, what got rejected and why, every timing that mattered.
• writeup/L1.md → L5.md — one report per level.
• writeup/L3_dashboard.html — open directly in a browser for the interactive aggregate-insight dashboard.
• outputs/L4_spatial_grounding/verify_map.html — the Leaflet ground-truth verification map.