feat(annotations): export COCO/YOLO 2D annotations from a scene manifest#69
Merged
Merged
Conversation
The manifest already records every point's per-camera uv plus in_view/visible, which is exactly what a 2D detection + keypoint training set needs. Add a typed exporter (annotations.py) that turns a Manifest into COCO keypoint JSON and per-image YOLO(-pose) TXT labels: one image per (camera, frame), entity ids as categories, and the COCO visibility flag (2 visible / 1 occluded / 0 out of view) derived from in_view/visible. Bounding boxes are the pixel bounds of an entity's in-view points, so they always lie within the frame. Typed all the way through with pydantic models, mirroring manifest.py; no optional extras. Round-trip test exports then parses back and checks the boxes match the manifest uv within tolerance, plus the visibility-flag mapping and the image/category structure. Closes bamdadd#40.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
The manifest already records, for every named point of every entity at every frame, each camera's pixel projection (
uv) plus thein_view/visibleflags — exactly the ground truth a 2D detection + keypoint training set needs. This adds a typed exporter so a simulated scene doubles as a synthetic training-data generator, not just a triangulation benchmark.Closes #40.
The exporter (
multicam_sim/annotations.py)export_coco(manifest) -> CocoDataset/write_coco(manifest, path)— COCO keypoint-detection JSON.export_yolo(manifest) -> YoloDataset/write_yolo(manifest, out_dir)— one YOLO(-pose).txtper image + aclasses.txt.Conventions:
(camera, frame)pair (file_name = cam{id}/frame_{frame:06d}.jpg, a logical reference since the sim doesn't render pixels).in_viewpoint on that image.in_viewpoints, so the box always lies within the camera frame (off-image points are excluded).2=in_viewandvisible,1=in_viewbut occluded,0= notin_view(absent,x = y = 0).edges) becomes the COCOskeleton(1-indexed pairs into that category's keypoints).[0, 1]by each image's width/height.Everything is typed all the way through with pydantic models, mirroring
manifest.py. Nothing here depends on an optional extra.Tests (
tests/test_annotations.py)uvwithin tolerance; COCO bboxes are checked against the manifestuvdirectly.in_view/visible(the pose scene's occludedleft_wriston camera 1 exercises the1flag; a synthetic manifest exercises the0/absent path and that it's excluded from the bbox).write_coco/write_yoloemit parseable files.Verification
ruff check,ruff format --check, andmypy --strict(28 src files) all clean.tests/test_smoke.pyetc. import the siblingmulticam-occlusionreader, which CI checks out at../multicam-occlusionbut isn't installed locally). This change doesn't touch them.README gains a short Quickstart snippet for the exporter.