Ball tracking for FTC that works with the camera you already have. Built and tested by students and teams in the community.
| Camera | How it works | Pick it if | Setup |
|---|---|---|---|
| Limelight 3A | A neural network runs on the camera's own processor | You want the steadiest tracking in matches | Guide |
| Webcam | Colour thresholding on the Control Hub, no model | You want to try it today or prototype | Guide |
Have both? Start with the Limelight.
 |
 |
| Neural detection Pollen and nectar, found live |
Auto-chase Limelight 3A, by 47 Beavers (13100) |
 |
 |
| Webcam chase By Owen Cassidy (17113, Hunga Munga) |
Spline pathing Limelight 3A + PedroPathing, by Ajay Govindarajan (22297, Infinibots) |
Everything lives in the docs.
- New here? Read the overview.
- Example OpModes: Limelight driving · Limelight pathing · Webcam driving
- Stuck? Ask in the Discord.
Webcam code example
BallDetectorPipeline baller = new BallDetectorPipeline();
VisionPortal portal = new VisionPortal.Builder()
.setCamera(hardwareMap.get(WebcamName.class, "Webcam 1"))
.setCameraResolution(new Size(1280, 720))
.addProcessor(baller)
.enableLiveView(true)
.build();
BallDetectorPipeline.BallBlob red = baller.bestOf(BallDetectorPipeline.BallColor.RED);
if (red != null) {
telemetry.addData("red", "cx=%.2f cy=%.2f", red.cxNorm, red.cyNorm);
}bestOf(...) returns null when nothing of that colour passes the checks. Otherwise it returns a BallBlob with cxNorm and cyNorm (0 to 1, relative to the screen). Colours: YELLOW, RED, BLUE.
v2 is fully 8-bit and about 6× faster than v1. It was trained on hard negatives like shadows, wiring and signage, so it sees fewer balls that aren't there. Class order and FTC-side code are unchanged from v1, so upgrading means swapping one file.
Reading the raw outputs? Only trust slots where
i < num_detections[0]. Anything past that is uninitialized memory.
Model details
| File | best_limelight3a_ssd_mobilenetv2_300x300_v2_int8_dets25.tflite |
| Architecture | SSD-MobileNetV2, 300 × 300 |
| Input | uint8 [1,300,300,3], scale 1/255 |
| Output | TFLite_Detection_PostProcess: num[1], scores[1,25], class_ids[1,25], boxes[1,25,4] (normalized ymin,xmin,ymax,xmax), float32 |
| Max detections | 25 |
| Size | 5.19 MB |
Why 25 detections? The model keeps the top-N boxes after score filtering. With N=10, a frame with 14 balls could only report 10. A cap of 25 gives crowded scenes room, and sparse frames don't change (about 4 detections per frame on average in validation).
How it was built
SSD-MobileNetV2 (300×300) was trained with the TensorFlow Object Detection API for 30,000 steps on an RTX 3000 Ada, using a class-weighted loss and baked-in NMS thresholds (score 0.3, IoU 0.6, 20 per class, 20 total). Final total loss was 0.1815, with 97.3% precision.
It was then exported to a SavedModel with a native uint8 input wrapper and converted to TFLite with full int8 quantization, calibrated on 512 frames sampled evenly across the training set. Weights and activations are 8-bit. Only the post-process outputs stay float32, which is what the Limelight 3A's neural detector expects.
Every release is checked with an output contract test, a latency benchmark, and a validation A/B against the previous model.
Dataset
14,409 labeled images (11,895 train, 2,514 validation): synthetic CAD renders of the game balls, real match and arena footage, and hard-negative frames.
Community datasets used:
| Dataset | Author | License |
|---|---|---|
| FTC-BioBuzz | Bluedarkup on Roboflow | MIT |
| Chunc Pollen | Arush's Workspace on Roboflow | CC BY 4.0 |
| ProBotiX v2 testing | Arush v2 testing |
|---|---|
 |
 |
| Recorded by Team 16765 ProBotiX | Recorded by Arush, FTC 23511 |
- v2.0.0: up to 25 detections per frame, full int8 (about 6× faster), a bigger and harder dataset, 97.3% precision. No code changes needed from v1.
- v1.1.0: retrained on about 10,000 images. Shadows and off-field objects are no longer detected as game elements.
Hive Vision was started by Harjas Sidhu, the founder and lead developer. Sid Shah is a co-developer.
Thanks to the teams who tested it: Arush (23511a), Ajay Govindarajan (22297), 47 Beavers (13100), Owen Cassidy (17113), ProBotiX (16765), Javi Nashat (23286) and Jasper Wolves Blue (11419). Their field reports are the reason v1.1 and v2 exist.
Join the Discord to share robot tests, training data, detector results and setup questions. Bugs, suggestions and dataset contributions can go in a GitHub issue.
- Updates: a release or model update is planned about every other week all season, with new weights, HSV settings, eval results and demo samples.
- Training data: share a link in the Discord's training-data discussion, with the source, camera resolution, frame rate, lighting and permission to use it. See the training data guide.
- Code: see CONTRIBUTING.md.
- Certificates: contributors whose data or testing work is accepted can request a certificate for their specific contribution, such as data collection, labeling, evaluation or detector testing. It's fine for a portfolio or resume, and it doesn't imply authorship of the whole project. Ask in an issue or the Discord.
Original Hive Vision code, documentation and configuration are MIT. See LICENSE. YOLO model artifacts and Ultralytics-dependent material are AGPL-3.0. See THIRD_PARTY_NOTICES.md.
