Skip to content

feat(annotations): export COCO/YOLO 2D annotations from a scene manifest#69

Merged
bamdadd merged 1 commit into
bamdadd:mainfrom
dchaudhari7177:feat/coco-yolo-exporter
Jul 24, 2026
Merged

feat(annotations): export COCO/YOLO 2D annotations from a scene manifest#69
bamdadd merged 1 commit into
bamdadd:mainfrom
dchaudhari7177:feat/coco-yolo-exporter

Conversation

@dchaudhari7177

Copy link
Copy Markdown
Contributor

What & why

The manifest already records, for every named point of every entity at every frame, each camera's pixel projection (uv) plus the in_view / visible flags — exactly the ground truth a 2D detection + keypoint training set needs. This adds a typed exporter so a simulated scene doubles as a synthetic training-data generator, not just a triangulation benchmark.

Closes #40.

The exporter (multicam_sim/annotations.py)

  • export_coco(manifest) -> CocoDataset / write_coco(manifest, path) — COCO keypoint-detection JSON.
  • export_yolo(manifest) -> YoloDataset / write_yolo(manifest, out_dir) — one YOLO(-pose) .txt per image + a classes.txt.

Conventions:

  • One image per (camera, frame) pair (file_name = cam{id}/frame_{frame:06d}.jpg, a logical reference since the sim doesn't render pixels).
  • One annotation per entity that has at least one in_view point on that image.
  • Bounding box = the axis-aligned pixel bounds of that entity's in_view points, so the box always lies within the camera frame (off-image points are excluded).
  • Keypoints follow the entity's point set with the COCO visibility flag derived from the manifest: 2 = in_view and visible, 1 = in_view but occluded, 0 = not in_view (absent, x = y = 0).
  • Categories are entity ids; the per-entity skeleton (edges) becomes the COCO skeleton (1-indexed pairs into that category's keypoints).
  • YOLO coordinates are normalised to [0, 1] by each image's width/height.

Everything is typed all the way through with pydantic models, mirroring manifest.py. Nothing here depends on an optional extra.

Tests (tests/test_annotations.py)

  • Round-trip (the issue's ask): export → parse the YOLO line back → denormalise → boxes match the manifest uv within tolerance; COCO bboxes are checked against the manifest uv directly.
  • Visibility-flag mapping tracks in_view / visible (the pose scene's occluded left_wrist on camera 1 exercises the 1 flag; a synthetic manifest exercises the 0/absent path and that it's excluded from the bbox).
  • Image coverage (every camera×frame pair, unique ids), category/skeleton structure, multi-entity categories, and that write_coco / write_yolo emit parseable files.

Verification

  • New tests: 9 passing.
  • Full collectable suite green; ruff check, ruff format --check, and mypy --strict (28 src files) all clean.
  • The 5 pre-existing collection errors in the suite are unrelated (tests/test_smoke.py etc. import the sibling multicam-occlusion reader, which CI checks out at ../multicam-occlusion but isn't installed locally). This change doesn't touch them.

README gains a short Quickstart snippet for the exporter.

The manifest already records every point's per-camera uv plus in_view/visible,
which is exactly what a 2D detection + keypoint training set needs. Add a typed
exporter (annotations.py) that turns a Manifest into COCO keypoint JSON and
per-image YOLO(-pose) TXT labels: one image per (camera, frame), entity ids as
categories, and the COCO visibility flag (2 visible / 1 occluded / 0 out of view)
derived from in_view/visible. Bounding boxes are the pixel bounds of an entity's
in-view points, so they always lie within the frame.

Typed all the way through with pydantic models, mirroring manifest.py; no
optional extras. Round-trip test exports then parses back and checks the boxes
match the manifest uv within tolerance, plus the visibility-flag mapping and the
image/category structure. Closes bamdadd#40.
@bamdadd
bamdadd merged commit e3ab82f into bamdadd:main Jul 24, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Export COCO / YOLO annotations from a scene manifest

2 participants