Skip to content

Per-entity posture / activity-state channel as labeled ground truth for activity recognition #65

Description

@bamdadd

Rationale

Multi-camera activity recognition needs a labeled per-entity, per-frame activity-state channel (e.g. standing / crouching / reaching) to score classifiers against. The manifest today carries geometry and occlusion but no discrete activity-state ground truth, so there is nothing to evaluate an activity head on.

Distinct from #29 (skeletal motion DSL). #29 generates joint motion (walk / reach / wave gaits) — it is a motion producer. This issue adds the labeled GT state channel that says which activity state each entity is in at each frame, so an activity-recognition eval has a target. The two compose: #29 can drive the joints, this issue labels the resulting state. This does not require #29.

What to build

A ground-truth channel that assigns each entity a discrete activity state per frame (or per interval), from a small open enum, emitted as opt-in GT.

Acceptance criteria

  • A typed activity-state label (small extensible enum, e.g. standing / crouching / reaching, open to more without a schema fork) attachable per entity over frame intervals.
  • Ground truth exposes, per entity per frame, its activity state (queryable timeline; contiguous intervals acceptable).
  • Additive only. The label channel is opt-in — either a sidecar (per order.py) or an opt-in None field — and the byte-golden manifest stays byte-identical when the channel is not requested.
  • Round-trips to JSON and reloads.
  • A test asserts the per-frame state timeline for a constructed scene.

Files / data model to extend

  • src/multicam_sim/entities.pyEntity / EntityFrame.
  • src/multicam_sim/pose.pyPoseTrajectory / Skeleton (a pose-driven entity is the natural carrier of a posture label).
  • src/multicam_sim/order.py — sidecar precedent if the channel is emitted as a sidecar.
  • Related: Skeletal motion DSL: walk / reach / wave gaits for the COCO-17 skeleton #29 (skeletal motion DSL) — motion producer this composes with.

Open design note

Sidecar vs. opt-in field for the label channel is left to the maintainer; the byte-golden constraint holds either way.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions