diff --git a/egomimic/rldb/mcap/SPEC.md b/egomimic/rldb/mcap/SPEC.md new file mode 100644 index 0000000000..45c525933e --- /dev/null +++ b/egomimic/rldb/mcap/SPEC.md @@ -0,0 +1,285 @@ +# EgoVerse vendor MCAP spec 0.1 + +What a data vendor delivers to EgoVerse: one raw, self-contained MCAP recording per +episode, from which we build the Lance tables (`recordings`, `tasks`, `annotations`, +`steps`) and the training videos, or any other format later, without asking the vendor +anything. + +- **Raw, not processed.** Original sensor data at its own rates: no resampling, no + aligning streams to one clock rate, no undistorting or cropping video. We convert on + our side. +- **Self-contained.** Calibration, camera positions and episode info live inside the + file. No side JSONs, no constants in our converters. +- **Checked on arrival.** Every delivered episode is run through the validator + (`python -m egomimic.rldb.mcap.validate`), which lists what is missing or wrong. + +The machine-readable version of this document is [`spec.py`](spec.py); the message +definitions are [`proto/egoverse.proto`](proto/egoverse.proto) (generated from +[`schemas.py`](schemas.py)). The three change together. + +Spec 0.1 covers **human (egocentric) recordings** and **Eva robot recordings with +grippers**. Dexterous hands, tactile, humanoid / auxiliary joints (e.g. Sharpa) are not +in 0.1. + +## 1. What arrives per episode + +``` +episode_/ +├── episode.mcap # everything: video, sensors, poses, calibration, episode info +└── annotation.mcap # optional: /task spans and /subtask-annotation labels +``` + +- **An episode is one whole recording session, never a pre-cut clip.** It becomes one + row of `recordings`; the tasks performed in it become `tasks` rows. +- The label topics may instead be inline in `episode.mcap` (as ABC-130k does), but not in + both files. `annotation.mcap` holds nothing but label topics: no metadata records + (the episode info lives in `episode.mcap` only). +- MCAP with protobuf encoding; any chunk compression (zstd, lz4) is fine. + +## 2. Clock + +- **Every message's MCAP log time is its capture time in absolute Unix nanoseconds**, in + both files. Devices that stamp on their own clock (e.g. Aria device time) convert + with their device-to-UTC offset before writing. +- Within a topic, log times never decrease; each camera's strictly increase (every + frame has its own time). +- `start-time-unix` / `end-time-unix` in the episode info are **milliseconds** and must + match the recorded data within 1 s. +- **Steps clock.** The main camera (`/main-camera`, human) or top camera (`/top-camera`, + robot) defines the Lance `steps` rows: one row per decoded frame. Every other stream is + aligned to those frame times during conversion; wrist and SLAM cameras stay separate + videos and add no rows. + +## 3. Who, what, where: episode info + +One MCAP metadata record named **`session-metadata`**. All values are strings. + +| Key | Example | Meaning | +|---|---|---| +| `lab` | `aria` | Data source / capture family; the Lance partition (`steps/.lance`). Slug. | +| `site` | `rl2` | Organization that collected the recording. Slug. | +| `vendor` | `acme` | Organization or pipeline that delivered / processed it. Slug. | +| `vendor-recording-id` | `acme-rec-0001` | The vendor's own ID for this recording. **Stable across every re-export or correction** (new labels, fixed calibration, re-run poses). | +| `operator-id` | `9f2c…e41a` | Who performed it, as a SHA-256 hash (64 lowercase hex digits). Never a name or email. | +| `task` | `fold_clothes` | Task slug; reuse an existing one if it fits. | +| `instruction` | `fold the 2T shirt on the blue table` | Full task text. | +| `objects` | `baby_shirt_2t,basket` | Objects involved, comma-separated slugs. | +| `embodiment` | `human_bimanual` | One of `human_{bimanual,left_arm,right_arm}`, `eva_{bimanual,left_arm,right_arm}`. Must match the topics in the file. | +| `rig-name` | `aria_gen1` | Capture setup. | +| `device-serial` | `1WM103600M1292` | Physical unit; ties the episode to its calibration. | +| `scene` | `kitchen_a` | Where it was recorded. | +| `start-time-unix`, `end-time-unix` | `1736651249664` | Recording bounds, Unix **ms**. | +| `spec-version` | `0.1` | Version of this spec the file follows. | +| `license` | `CC-BY-4.0` | License the data is under. | +| `camera-timestamp-ref` | `mid` | *Human only.* Where in the exposure each frame is stamped: `start`, `mid` or `end`. | +| `shutter` | `rolling:5ms` | *Human only.* `global`, or `rolling:ms`. | +| `hand-joints` | see §6 | *Human only.* Names of the hand joints, in the order `HandKeypoints.positions` lists them. | + +Slugs are lowercase letters and digits, words joined by `_` or `-`. + +**IDs.** Our recording ID is +`egomimic.rldb.lance.schema.recording_id(lab, vendor-recording-id)` (UUIDv5 of +`/`); the validator prints it. Task and annotation IDs are +computed downstream from their content, so vendors send none. + +**Tasks.** Without `/task` topics, `task` / `instruction` / `objects` describe one task +spanning the whole recording: [first steps frame, last steps frame + one frame), as +every span is half-open. With `/task`, the spans are authoritative and these keys +only summarize the session (e.g. `task=laundry` over iron / fold / pack spans); no extra +task is made from them. + +## 4. Topics + +Message types are Foxglove's (`foxglove-schemas-protobuf`) where one exists, otherwise +`egoverse.*` from `proto/egoverse.proto`. Topic names, message types and required fields +are checked exactly. + +### Human / egocentric + +| Topic | Type | | Rate | Notes | +|---|---|---|---|---| +| `/main-camera` | `foxglove.CompressedVideo` | MUST | ≥ 30 fps | Head-mounted RGB; the steps clock. | +| `/main-camera-info` | `foxglove.CameraCalibration` | MUST | once | See §7. | +| `/slam-left-camera`, `/slam-right-camera` | `foxglove.CompressedVideo` | optional | ≥ 30 fps | Each with its `-info`. | +| `/left-wrist-camera`, `/right-wrist-camera` | `foxglove.CompressedVideo` | optional | ≥ 30 fps | Each with its `-info`. | +| `/imu` | `egoverse.Imu` | MUST | ≥ 200 Hz | Raw: gravity included, no fusion. | +| `/imu-info` | `egoverse.ImuInfo` | MUST | once | Noise model; datasheet values are fine. | +| `/head-pose` | `foxglove.PoseInFrame` | MUST | ≥ 30 Hz | `frame_id = "world"`, pose = T_world_head. | +| `/head-pose-quality` | `egoverse.TrackingQuality` | MUST | one per head pose | `OK`, `LOST`, `RELOCALIZED` (first pose after a SLAM reset) or `UNKNOWN`. | +| `/{left,right}-hand-keypoints` | `egoverse.HandKeypoints` | MUST per arm in use | tracker rate | See §6. | +| `/eye-gaze` | `egoverse.EyeGaze` | optional | tracker rate | Yaw / pitch / depth in a frame between the eyes (+x left, +y up, +z forward). | +| `/tf_static` | `foxglove.FrameTransforms` | MUST | once | See §5. | +| `/instruction` | `egoverse.Instruction` | MUST | once | Task text. | + +No arm, gripper or action topics: actions are derived from future hand and head poses. + +### Robot (Eva with grippers) + +`{side}` is `left` and/or `right`, as the embodiment uses. + +| Topic | Type | | Notes | +|---|---|---|---| +| `/top-camera` | `foxglove.CompressedVideo` | MUST | ≥ 30 fps; the steps clock. With `-info`. | +| `/{side}-wrist-camera` | `foxglove.CompressedVideo` | MUST | ≥ 30 fps. With `-info`. | +| `/{side}-arm-state` | `egoverse.ArmState` | MUST | Measured joints (names, position, velocity, torque) and `ee_pose`. | +| `/{side}-arm-action` | `egoverse.ArmAction` | MUST | The command sent at that time: target joints and `ee_pose`. | +| `/{side}-ee-state` | `egoverse.GripperState` | MUST | Measured opening, 0 = closed, 1 = fully open. | +| `/{side}-ee-action` | `egoverse.GripperState` | MUST | Commanded opening, same range. | +| `/tf_static` | `foxglove.FrameTransforms` | MUST | See §5. | +| `/instruction` | `egoverse.Instruction` | MUST | Task text. | + +`ArmState` / `ArmAction`: `frame_id` is the arm-base frame (`left_arm_base`, +`right_arm_base`); every joint array has one entry per `joint_names` entry, in that +order; joints in rad (m for prismatic), velocities rad/s, torques N·m; `ee_pose` is +T_base_ee as 16 numbers, a **row-major** 4×4. ABC-130k's own `RobotState` / +`GripperState` are converted to these types before validation. + +### Both profiles: tasks and labels (optional) + +| Topic | Type | Notes | +|---|---|---| +| `/task` | `egoverse.Task` | One message per task performed: `name` (slug), `description`, `objects`; starts at its log time, ends at `end_time_unix_ns` (after the start). | +| `/subtask-annotation` | `egoverse.Annotation` | A language label `data` over [log time, `end_time_unix_ns`); `end_time_unix_ns = 0` is a point event (one frame wide). `annotation_type` is `language` (the default when empty); `annotation_vendor` defaults to `vendor`. | + +Every label lies **inside one task**; without `/task`, inside the whole-recording task +made from the episode info. All spans lie inside the recorded data. Labels added or corrected after delivery go through +the Lance annotation review (`egomimic/rldb/lance/annotations.py`), not a re-export. + +## 5. Frames and conventions + +- Right-handed frames; metres, radians, seconds. +- Quaternions are Foxglove's named `x, y, z, w` fields and must be unit length. +- A transform or pose is **T_parent_child**: the child frame's pose in the parent + (Foxglove's `FrameTransform` / `PoseInFrame` convention). +- Every camera frame is a Foxglove optical frame: +x right, +y down, +z forward, in the + raw sensor orientation (see `upright_rotation_cw_deg`, §7). A camera's video, + `-info` and `-info-extra` all name that one frame. +- **Every frame any topic names must reach `world`** through `/tf_static` plus the + moving links the profile defines: + - **Human:** `world → head` moves via `/head-pose`. `/tf_static` links `head` to each + camera, the IMU, the gaze frame, and any frame hands are given in. The main camera + must be rigidly mounted on the head: a `/tf_static` path from `head` to its frame. + - **Robot:** `world → {side}_arm_base` and the top camera in `/tf_static`; + `{side}_arm_base → {side}_ee` moves via `/{side}-arm-state`; the wrist camera hangs + off `{side}_ee` in `/tf_static`. + +## 6. Hands + +`HandKeypoints`, one message per hand per tracker frame: + +| Field | | +|---|---| +| `frame_id` | Frame `positions` and `wrist_pose` are in (linked to `world`). | +| `positions` | `(x, y, z)` in m per `hand-joints` entry, flattened; or empty for devices without keypoints. | +| `confidence` | Whole-hand tracking confidence in [0, 1]. | +| `joint_confidence` | Optional: one value in [0, 1] per joint. | +| `wrist_pose` | Optional: T_frame_wrist. Required when `positions` is empty. | +| `valid` | `false` when the hand is not tracked. **Never write zeros instead.** | + +**`hand-joints`** names the joints the device tracks, in its own order, from this +vocabulary: the 21 MANO joints `wrist`, `{thumb}_{cmc,mcp,ip,tip}`, +`{index,middle,ring,pinky}_{mcp,pip,dip,tip}`, plus `palm_center`. Each name at most +once; `wrist` is required. Joints with MANO names are what our MANO fit matches; extras +are kept but not fitted. A device that already outputs MANO sends the canonical order: + +``` +wrist,thumb_cmc,thumb_mcp,thumb_ip,thumb_tip,index_mcp,index_pip,index_dip,index_tip, +middle_mcp,middle_pip,middle_dip,middle_tip,ring_mcp,ring_pip,ring_dip,ring_tip, +pinky_mcp,pinky_pip,pinky_dip,pinky_tip +``` + +and a device with its own layout lists it as is, e.g. Aria MPS's 21 landmarks (no thumb +CMC, a palm center instead): + +``` +thumb_tip,index_tip,middle_tip,ring_tip,pinky_tip,wrist,thumb_mcp,thumb_ip, +index_mcp,index_pip,index_dip,middle_mcp,middle_pip,middle_dip,ring_mcp,ring_pip, +ring_dip,pinky_mcp,pinky_pip,pinky_dip,palm_center +``` + +## 7. Cameras + +- **Video.** H.264 or H.265, Annex B byte stream, `CompressedVideo.format` `h264` / + `h265`. **Each message is exactly one frame**, stamped with that frame's capture time; + the first message is a keyframe and carries SPS/PPS (repeat them at every keyframe). + Original resolution and orientation: no resizing, cropping, undistorting or rotating. +- **Calibration** `-info`: exactly one `foxglove.CameraCalibration` message, + finite values, principal point inside the image: + `width` / `height` equal the decoded frames; `K` has fx, fy > 0. Name the lens model in + `distortion_model` and give exactly its number of coefficients in `D` (an empty model + with no coefficients is a pinhole camera): + + | `distortion_model` | `K` | `D` | + |---|---|---| + | `plumb_bob` | fx, fy, cx, cy | k1, k2, p1, p2, k3 | + | `rational_polynomial` | fx, fy, cx, cy | k1, k2, p1, p2, k3, k4, k5, k6 | + | `kannala_brandt` | fx, fy, cx, cy | k1, k2, k3, k4 | + | `fisheye624` (Aria) | f, f, cx, cy | the 12 coefficients after f, cx, cy, in projectaria order (k0–k5, p0, p1, s0–s3) | + +- **Optional** `-info-extra`, `egoverse.CameraInfoExtra`, published once: + `valid_radius` (px from the principal point where the lens model holds; 0 = whole + image) and `upright_rotation_cw_deg` (0, 90, 180 or 270: the clockwise turn that makes + a frame upright, e.g. 90 for Aria's sideways RGB sensor). + +## 8. Please don't + +- Re-encode, resize, crop, rotate or undistort video. *(Devices that store frames as + images, e.g. Aria VRS, encode them once to H.264 / H.265 at full resolution.)* +- Pack several frames into one video message. +- Resample or align streams to one rate; we match by timestamp. +- Trim streams to the camera's span; deliver every sample. +- Write zeros when tracking is lost; set `valid = false`. +- Remove gravity from, filter or fuse the IMU data. +- Put real operator names or emails anywhere; hash them. +- Ship calibration as a separate file. + +## 9. How a delivery is checked + +`python -m egomimic.rldb.mcap.validate episode_/ [...]` prints `PASS` or every +problem per episode (with the episode's `recording_id`), and exits 1 if any fails. Every +problem fails the episode. + +| Group | Checks | +|---|---| +| everything there | Required topics present with the exact type and encoding; required fields non-empty on every message; every episode-info key present, slugs / integers / enums valid; `embodiment` matches the topics; `annotation.mcap` holds only label topics. | +| video usable | Codec h264 / h265; first message a keyframe; one frame per message; frames decode; size matches the calibration; ≥ 30 fps. | +| clocks | Unix ns (after 2015); never decreasing; no gap over 0.5 s; streams cover the recording to within 1 s; start / end times match the data to within 1 s; head pose and quality ≥ 30 Hz. | +| calibration | Every camera has a valid `-info` with a known lens model and its coefficient count; `-info-extra` values valid; a camera's video and calibration name the same frame; every named frame reaches `world`; a head-worn rig's main camera has a `/tf_static` path to `head`; `/tf_static` rotations unit and translations finite. | +| imu | ≥ 200 Hz; mean \|accel\| in the stillest 1 s within 0.5 m/s² of 9.81 (gravity not removed); noise values > 0. | +| poses | No NaN / Inf, unit rotations (head, wrist, `/tf_static`); hand positions match `hand-joints`; confidences in [0, 1]; no all-zero hands marked valid; gaze finite when valid; `ee_pose` a rigid 4×4; joint arrays match `joint_names`; grippers in [0, 1]. | +| tasks & labels | Spans end after they start (labels may be points); inside the recorded data; each label inside one task; `annotation_type` known. | + +Thresholds stated by the plan (30 fps, 200 Hz, 30 Hz, 9.81 m/s²) are fixed. The others +(5 % rate tolerance, 0.5 s gap, 1 s span tolerance, 0.5 m/s² gravity tolerance, 1e-3 +unit tolerance) are **provisional, to be tuned on real deliveries**; all live in +`spec.py`. Video is decoded for the first 60 messages of each camera; the converter then decodes +all of it and rejects an episode where any message is not exactly one frame. + +## 10. Where each Lance field comes from + +| Lance | MCAP | +|---|---| +| `recordings.recording_id` | `recording_id(lab, vendor-recording-id)` | +| `recordings.vendor_recording_id`, `lab`, `site`, `embodiment`, `rig_name`, `scene`, `license` | episode info | +| `recordings.operator` | `operator-id` | +| `recordings.captured_at` | `start-time-unix` | +| `recordings.num_frames`, `fps` | Decoded steps-clock frames and their timestamps | +| `recordings.video_path` | Assigned when each camera topic is extracted | +| `recordings.intrinsics` / `extrinsics` | Each camera's `-info` (+ `-info-extra`) / `/tf_static` and the moving links | +| `tasks.*` | `/task` messages, or the episode-info task keys over the whole recording | +| `annotations.*` | `/subtask-annotation` (type, text, vendor, span) | +| `steps.timestamp_ns` | Steps-clock frame times | +| `steps.head_pose`, `head_pose_valid` | `/head-pose` (× the camera's link), `/head-pose-quality`, timing gaps | +| `steps.{side}.keypoints`, `ee_pose` | `HandKeypoints.positions`, through the MANO fit when `hand-joints` is not canonical MANO | +| `steps.{side}.wrist_pose`, `confidence`, `{side}_valid` | `HandKeypoints.wrist_pose`, `confidence`, `valid` | +| `steps.eye_gaze` | `/eye-gaze` yaw, pitch, depth | +| Eva steps `obs_*` / `cmd_*` | `/{side}-arm-state` / `-arm-action` (`position`, `ee_pose`), `/{side}-ee-state` / `-ee-action` | + +## 11. Not in 0.1 + +Deliberately left out of this version, to revisit with real deliveries: + +- Graded SLAM quality (Aria MPS gives 0 / 0.5 / 1), beyond the `TrackingQuality` states. +- More than one IMU per device. +- Video from devices that store per-frame images: re-encode (as now) or accept them. +- Dexterous hands, tactile and auxiliary joints (Sharpa). diff --git a/egomimic/rldb/mcap/__init__.py b/egomimic/rldb/mcap/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/egomimic/rldb/mcap/proto/egoverse.proto b/egomimic/rldb/mcap/proto/egoverse.proto new file mode 100644 index 0000000000..a6a4012fb2 --- /dev/null +++ b/egomimic/rldb/mcap/proto/egoverse.proto @@ -0,0 +1,137 @@ +// Generated by `python -m egomimic.rldb.mcap.schemas`; do not edit by hand. +// EgoVerse vendor spec 0.1: message types not covered by foxglove schemas. + +syntax = "proto3"; + +package egoverse; + +// A 3D vector. +message Vector3 { + double x = 1; + double y = 2; + double z = 3; +} + +// A rotation; must be unit length. +message Quaternion { + double x = 1; + double y = 2; + double z = 3; + double w = 4; +} + +// A child frame's pose in its parent (T_parent_child): m and a unit quaternion. +message Pose { + Vector3 position = 1; + Quaternion orientation = 2; +} + +// Optional per-camera facts foxglove.CameraCalibration has no field for, on the +// camera's -info-extra, published once next to -info. +message CameraInfoExtra { + string frame_id = 1; // the camera's frame, as in its -info + double valid_radius = 2; // px from the principal point where the lens model holds (fisheye); 0 = the whole image + int32 upright_rotation_cw_deg = 3; // clockwise turn (0, 90, 180, 270) that makes a frame upright, for sensors mounted sideways +} + +// One raw IMU sample, straight off the device: no gravity removal, no fusion. +message Imu { + string frame_id = 1; // IMU frame; must be linked in /tf_static + Vector3 linear_acceleration = 2; // m/s^2, gravity included + Vector3 angular_velocity = 3; // rad/s +} + +// IMU noise model, as SLAM/VIO tools expect it. Published once. Datasheet values +// are fine when the device does not calibrate them. +message ImuInfo { + string frame_id = 1; + double accel_noise_density = 2; // m/s^2/sqrt(Hz) + double accel_random_walk = 3; // m/s^3/sqrt(Hz) + double gyro_noise_density = 4; // rad/s/sqrt(Hz) + double gyro_random_walk = 5; // rad/s^2/sqrt(Hz) +} + +// State of the vendor's SLAM at this time, one per head pose. +message TrackingQuality { + enum State { + UNKNOWN = 0; + OK = 1; + LOST = 2; + RELOCALIZED = 3; + } + State state = 1; +} + +// One hand at one time: its joints and/or the wrist pose. Joints are in the order +// the episode's hand-joints key lists them (names from spec.HAND_JOINTS). Set +// valid = false when the hand is not tracked; never write zeros in its place. +// A device without keypoints leaves positions empty and sets wrist_pose. +message HandKeypoints { + string frame_id = 1; // frame positions and wrist_pose are in + repeated double positions = 2; // (x, y, z) in m per hand-joints entry, flattened; or empty + float confidence = 3; // whole-hand tracking confidence in [0, 1] + bool valid = 4; // hand tracked and its values usable + repeated float joint_confidence = 5; // optional: per-joint confidences in [0, 1], one per joint + Pose wrist_pose = 6; // optional: T_frame_wrist +} + +// Where the wearer looks, as in Aria MPS eye gaze. frame_id is centered between +// the eyes with +x to the wearer's left, +y up, +z forward; the gaze point is +// depth * normalize(tan(yaw), tan(pitch), 1). Set valid = false when gaze is not +// tracked; never write zeros in its place. +message EyeGaze { + string frame_id = 1; // gaze frame; linked in /tf_static + double yaw = 2; // rad, positive = looking left + double pitch = 3; // rad, positive = looking up + double depth = 4; // m, distance to the gaze point + bool valid = 5; // gaze tracked and values usable +} + +// Measured state of one arm. Joint arrays follow joint_names. ee_pose is +// T_base_ee: the end effector in the arm-base frame (frame_id), row-major 4x4. +message ArmState { + string frame_id = 1; // arm-base frame, e.g. "left_arm_base" + repeated string joint_names = 2; + repeated double position = 3; // rad (or m, prismatic) + repeated double velocity = 4; // rad/s (or m/s) + repeated double torque = 5; // N*m (or N) + repeated double ee_pose = 6; // 16 values, row-major +} + +// Command sent to one arm at this time (the target it was told to reach), +// same conventions as ArmState. +message ArmAction { + string frame_id = 1; // arm-base frame + repeated string joint_names = 2; + repeated double position = 3; // rad (or m, prismatic) + repeated double ee_pose = 4; // 16 values, row-major +} + +// Gripper opening, measured (ee-state) or commanded (ee-action). +message GripperState { + double position = 1; // 0 = closed, 1 = fully open +} + +// One task performed in the recording; a recording may hold several +// (e.g. iron, then fold, then pack). Starts at the message's log time. +message Task { + string name = 1; // task slug, e.g. "fold_clothes" + string description = 2; // free text for this instance + repeated string objects = 3; // object slugs involved + int64 end_time_unix_ns = 4; // task end, absolute Unix ns +} + +// A label over [start, end), start being the message's log time; it must lie +// inside one /task (or the recording, without /task). Field `data` matches +// ABC-130k's /subtask-annotation. +message Annotation { + string data = 1; // English text + int64 end_time_unix_ns = 2; // end, absolute Unix ns; 0 = point event (one frame wide) + string annotation_type = 3; // default "language" + string annotation_vendor = 4; // who wrote it; default the episode's vendor +} + +// Task text for the episode, e.g. "fold the shirt and put it in the basket". +message Instruction { + string data = 1; +} diff --git a/egomimic/rldb/mcap/schemas.py b/egomimic/rldb/mcap/schemas.py new file mode 100644 index 0000000000..dfbd92fd15 --- /dev/null +++ b/egomimic/rldb/mcap/schemas.py @@ -0,0 +1,320 @@ +"""EgoVerse's own MCAP message types (vendor spec 0.1). + +Only what Foxglove's schemas do not cover. Video, camera calibration, transforms and head +pose use `foxglove.CompressedVideo`, `foxglove.CameraCalibration`, `foxglove.FrameTransforms` +and `foxglove.PoseInFrame` (package `foxglove-schemas-protobuf`). Robot vendors use the +ArmState / ArmAction / GripperState below; ABC-130k's own robot types are normalized to +them before validation. + +The messages are declared once, here, and built into a descriptor pool at import time, so +no `protoc` step is needed. `render_proto()` prints the same definitions as the +`proto/egoverse.proto` file handed to vendors; a unit test keeps the two in sync. + +A message's time is its MCAP log time (absolute Unix ns), so none of them carry a timestamp; +spans (Task, Annotation) add only their end time. + + python -m egomimic.rldb.mcap.schemas > egomimic/rldb/mcap/proto/egoverse.proto +""" + +from dataclasses import dataclass, field + +from google.protobuf import descriptor_pb2, descriptor_pool, message_factory + +PACKAGE = "egoverse" +PROTO_FILE = "egoverse.proto" + +_F = descriptor_pb2.FieldDescriptorProto +_SCALARS = { + "double": _F.TYPE_DOUBLE, + "float": _F.TYPE_FLOAT, + "int64": _F.TYPE_INT64, + "string": _F.TYPE_STRING, + "int32": _F.TYPE_INT32, + "bool": _F.TYPE_BOOL, +} + + +@dataclass(frozen=True) +class Field: + name: str + kind: str # a scalar above, an enum of the same message, or another message here + repeated: bool = False + doc: str = "" + + +@dataclass(frozen=True) +class Enum: + name: str + values: tuple[ + str, ... + ] # numbered 0, 1, ...; proto3 needs the 0 value to mean "unset" + + +@dataclass(frozen=True) +class Message: + name: str + doc: str + fields: tuple[Field, ...] + enums: tuple[Enum, ...] = field(default=()) + + +MESSAGES = ( + Message( + "Vector3", + "A 3D vector.", + (Field("x", "double"), Field("y", "double"), Field("z", "double")), + ), + Message( + "Quaternion", + "A rotation; must be unit length.", + ( + Field("x", "double"), + Field("y", "double"), + Field("z", "double"), + Field("w", "double"), + ), + ), + Message( + "Pose", + "A child frame's pose in its parent (T_parent_child): m and a unit quaternion.", + (Field("position", "Vector3"), Field("orientation", "Quaternion")), + ), + Message( + "CameraInfoExtra", + "Optional per-camera facts foxglove.CameraCalibration has no field for, on the\n" + "camera's -info-extra, published once next to -info.", + ( + Field("frame_id", "string", doc="the camera's frame, as in its -info"), + Field( + "valid_radius", + "double", + doc="px from the principal point where the lens model holds (fisheye); " + "0 = the whole image", + ), + Field( + "upright_rotation_cw_deg", + "int32", + doc="clockwise turn (0, 90, 180, 270) that makes a frame upright, for " + "sensors mounted sideways", + ), + ), + ), + Message( + "Imu", + "One raw IMU sample, straight off the device: no gravity removal, no fusion.", + ( + Field("frame_id", "string", doc="IMU frame; must be linked in /tf_static"), + Field("linear_acceleration", "Vector3", doc="m/s^2, gravity included"), + Field("angular_velocity", "Vector3", doc="rad/s"), + ), + ), + Message( + "ImuInfo", + "IMU noise model, as SLAM/VIO tools expect it. Published once. Datasheet values\n" + "are fine when the device does not calibrate them.", + ( + Field("frame_id", "string"), + Field("accel_noise_density", "double", doc="m/s^2/sqrt(Hz)"), + Field("accel_random_walk", "double", doc="m/s^3/sqrt(Hz)"), + Field("gyro_noise_density", "double", doc="rad/s/sqrt(Hz)"), + Field("gyro_random_walk", "double", doc="rad/s^2/sqrt(Hz)"), + ), + ), + Message( + "TrackingQuality", + "State of the vendor's SLAM at this time, one per head pose.", + (Field("state", "State"),), + enums=(Enum("State", ("UNKNOWN", "OK", "LOST", "RELOCALIZED")),), + ), + Message( + "HandKeypoints", + "One hand at one time: its joints and/or the wrist pose. Joints are in the order\n" + "the episode's hand-joints key lists them (names from spec.HAND_JOINTS). Set\n" + "valid = false when the hand is not tracked; never write zeros in its place.\n" + "A device without keypoints leaves positions empty and sets wrist_pose.", + ( + Field("frame_id", "string", doc="frame positions and wrist_pose are in"), + Field( + "positions", + "double", + repeated=True, + doc="(x, y, z) in m per hand-joints entry, flattened; or empty", + ), + Field( + "confidence", + "float", + doc="whole-hand tracking confidence in [0, 1]", + ), + Field("valid", "bool", doc="hand tracked and its values usable"), + Field( + "joint_confidence", + "float", + repeated=True, + doc="optional: per-joint confidences in [0, 1], one per joint", + ), + Field("wrist_pose", "Pose", doc="optional: T_frame_wrist"), + ), + ), + Message( + "EyeGaze", + "Where the wearer looks, as in Aria MPS eye gaze. frame_id is centered between\n" + "the eyes with +x to the wearer's left, +y up, +z forward; the gaze point is\n" + "depth * normalize(tan(yaw), tan(pitch), 1). Set valid = false when gaze is not\n" + "tracked; never write zeros in its place.", + ( + Field("frame_id", "string", doc="gaze frame; linked in /tf_static"), + Field("yaw", "double", doc="rad, positive = looking left"), + Field("pitch", "double", doc="rad, positive = looking up"), + Field("depth", "double", doc="m, distance to the gaze point"), + Field("valid", "bool", doc="gaze tracked and values usable"), + ), + ), + Message( + "ArmState", + "Measured state of one arm. Joint arrays follow joint_names. ee_pose is\n" + "T_base_ee: the end effector in the arm-base frame (frame_id), row-major 4x4.", + ( + Field("frame_id", "string", doc='arm-base frame, e.g. "left_arm_base"'), + Field("joint_names", "string", repeated=True), + Field("position", "double", repeated=True, doc="rad (or m, prismatic)"), + Field("velocity", "double", repeated=True, doc="rad/s (or m/s)"), + Field("torque", "double", repeated=True, doc="N*m (or N)"), + Field("ee_pose", "double", repeated=True, doc="16 values, row-major"), + ), + ), + Message( + "ArmAction", + "Command sent to one arm at this time (the target it was told to reach),\n" + "same conventions as ArmState.", + ( + Field("frame_id", "string", doc="arm-base frame"), + Field("joint_names", "string", repeated=True), + Field("position", "double", repeated=True, doc="rad (or m, prismatic)"), + Field("ee_pose", "double", repeated=True, doc="16 values, row-major"), + ), + ), + Message( + "GripperState", + "Gripper opening, measured (ee-state) or commanded (ee-action).", + (Field("position", "double", doc="0 = closed, 1 = fully open"),), + ), + Message( + "Task", + "One task performed in the recording; a recording may hold several\n" + "(e.g. iron, then fold, then pack). Starts at the message's log time.", + ( + Field("name", "string", doc='task slug, e.g. "fold_clothes"'), + Field("description", "string", doc="free text for this instance"), + Field("objects", "string", repeated=True, doc="object slugs involved"), + Field("end_time_unix_ns", "int64", doc="task end, absolute Unix ns"), + ), + ), + Message( + "Annotation", + "A label over [start, end), start being the message's log time; it must lie\n" + "inside one /task (or the recording, without /task). Field `data` matches\n" + "ABC-130k's /subtask-annotation.", + ( + Field("data", "string", doc="English text"), + Field( + "end_time_unix_ns", + "int64", + doc="end, absolute Unix ns; 0 = point event (one frame wide)", + ), + Field("annotation_type", "string", doc='default "language"'), + Field( + "annotation_vendor", + "string", + doc="who wrote it; default the episode's vendor", + ), + ), + ), + Message( + "Instruction", + 'Task text for the episode, e.g. "fold the shirt and put it in the basket".', + (Field("data", "string"),), + ), +) + + +def file_descriptor_proto() -> descriptor_pb2.FileDescriptorProto: + fp = descriptor_pb2.FileDescriptorProto( + name=PROTO_FILE, package=PACKAGE, syntax="proto3" + ) + for m in MESSAGES: + mp = fp.message_type.add(name=m.name) + for e in m.enums: + ep = mp.enum_type.add(name=e.name) + for number, value in enumerate(e.values): + ep.value.add(name=value, number=number) + enum_names = {e.name for e in m.enums} + for number, f in enumerate(m.fields, start=1): + label = _F.LABEL_REPEATED if f.repeated else _F.LABEL_OPTIONAL + fd = mp.field.add(name=f.name, number=number, label=label) + if f.kind in _SCALARS: + fd.type = _SCALARS[f.kind] + elif f.kind in enum_names: + fd.type = _F.TYPE_ENUM + fd.type_name = f".{PACKAGE}.{m.name}.{f.kind}" + else: + fd.type = _F.TYPE_MESSAGE + fd.type_name = f".{PACKAGE}.{f.kind}" + return fp + + +# Own pool, so importing this module never collides with another egoverse.proto. +_pool = descriptor_pool.DescriptorPool() +_pool.Add(file_descriptor_proto()) + + +def _message_class(name: str): + return message_factory.GetMessageClass( + _pool.FindMessageTypeByName(f"{PACKAGE}.{name}") + ) + + +Vector3 = _message_class("Vector3") +Quaternion = _message_class("Quaternion") +Pose = _message_class("Pose") +CameraInfoExtra = _message_class("CameraInfoExtra") +Imu = _message_class("Imu") +ImuInfo = _message_class("ImuInfo") +TrackingQuality = _message_class("TrackingQuality") +HandKeypoints = _message_class("HandKeypoints") +EyeGaze = _message_class("EyeGaze") +ArmState = _message_class("ArmState") +ArmAction = _message_class("ArmAction") +GripperState = _message_class("GripperState") +Task = _message_class("Task") +Annotation = _message_class("Annotation") +Instruction = _message_class("Instruction") + + +def render_proto() -> str: + """The `.proto` text for MESSAGES, as handed to vendors.""" + lines = [ + "// Generated by `python -m egomimic.rldb.mcap.schemas`; do not edit by hand.", + "// EgoVerse vendor spec 0.1: message types not covered by foxglove schemas.", + "", + 'syntax = "proto3";', + "", + f"package {PACKAGE};", + ] + for m in MESSAGES: + lines.append("") + lines += [f"// {line}" for line in m.doc.split("\n")] + lines.append(f"message {m.name} {{") + for e in m.enums: + lines.append(f" enum {e.name} {{") + lines += [f" {v} = {n};" for n, v in enumerate(e.values)] + lines.append(" }") + for number, f in enumerate(m.fields, start=1): + prefix = "repeated " if f.repeated else "" + comment = f" // {f.doc}" if f.doc else "" + lines.append(f" {prefix}{f.kind} {f.name} = {number};{comment}") + lines.append("}") + return "\n".join(lines) + "\n" + + +if __name__ == "__main__": + print(render_proto(), end="") diff --git a/egomimic/rldb/mcap/spec.py b/egomimic/rldb/mcap/spec.py new file mode 100644 index 0000000000..d28cf3a8c6 --- /dev/null +++ b/egomimic/rldb/mcap/spec.py @@ -0,0 +1,363 @@ +"""Vendor MCAP spec 0.1 in machine-readable form: what `validate.py` checks against. + +SPEC.md is the human-readable version of this file; change both together. Numbers the +plan states are used as-is; the ones it does not state are marked PROVISIONAL, to be tuned +on real deliveries. + +An episode is one whole recording session, never a pre-cut clip: it becomes one row of +the Lance `recordings` table, and the tasks inside it become `tasks` rows. + + episode_/ + ├── episode.mcap # everything: video, sensors, poses, calibration, episode info + └── annotation.mcap # optional: /task spans and /subtask-annotation labels + +Every message is stamped with absolute Unix time in nanoseconds (its MCAP log time), in +both files. Every camera is a topic ending in `-camera`; its calibration is on the sibling +`-info`. Each video message holds exactly one frame, stamped with its capture time. + +The profile's `step_clock` camera defines the Lance steps rows: one row per decoded frame +of it, every other stream aligned to those times during conversion. Other cameras stay +separate videos and add no rows. + +Who and where, as in the Lance pilot: `lab` is the data source / capture family and the +Lance partition (steps/.lance: aria, eva, mecka, scale), `site` the organization that +collected the recording (rl2, microagi), `vendor` who delivered or processed it (acme). +Hardware goes in rig-name / device-serial. All three are normalized slugs. + +IDs: Lance recording_id = recording_id(lab, vendor-recording-id) below, so +vendor-recording-id must stay the same across every re-export or correction of the +recording. Task and annotation IDs are computed downstream from their content. +""" + +import re +import uuid +from dataclasses import dataclass + +SPEC_VERSION = "0.1" +SUPPORTED_SPEC_VERSIONS = ("0.1",) + +EPISODE_FILE = "episode.mcap" +# Its topics may instead be inline in episode.mcap, as ABC-130k does. +ANNOTATION_FILE = "annotation.mcap" +# ABC's delivery format names its episode-info record this, with hyphenated keys. +METADATA_RECORD = "session-metadata" + +CAMERA_SUFFIX = "-camera" +CALIBRATION_SUFFIX = "-info" +# Optional egoverse.CameraInfoExtra (valid radius, upright rotation) next to the -info. +CAMERA_EXTRA_SUFFIX = "-info-extra" +UPRIGHT_ROTATIONS = (0, 90, 180, 270) +# CameraCalibration.distortion_model -> number of D coefficients. An empty model with no +# coefficients is a pinhole camera. +DISTORTION_MODELS = { + "plumb_bob": 5, + "rational_polynomial": 8, + "kannala_brandt": 4, + "fisheye624": 12, +} + +# Hand joint vocabulary. Vendors list the joints their device tracks, in the order they +# write them, in the "hand-joints" key. Names that are MANO joints are what the MANO fit +# (egomimic/scripts/aria_process/aria_utils.py) matches; extras are kept but not fitted. +# Canonical MANO order, as Human.FINGER_EDGES uses it: a vendor sending this exactly +# needs no fit. +MANO_JOINTS = ( + "wrist", + *( + f"{finger}_{joint}" + for finger, joints in ( + ("thumb", ("cmc", "mcp", "ip", "tip")), + ("index", ("mcp", "pip", "dip", "tip")), + ("middle", ("mcp", "pip", "dip", "tip")), + ("ring", ("mcp", "pip", "dip", "tip")), + ("pinky", ("mcp", "pip", "dip", "tip")), + ) + for joint in joints + ), +) +EXTRA_HAND_JOINTS = ("palm_center",) +HAND_JOINTS = MANO_JOINTS + EXTRA_HAND_JOINTS +# Shutter: global, or rolling with its readout time. +SHUTTER = re.compile(r"global|rolling:[0-9]+(\.[0-9]+)?ms") + +# Lowercase letters and digits, words joined by "_" or "-": safe in Lance paths and IDs. +SLUG = re.compile(r"[a-z0-9]+(?:[_-][a-z0-9]+)*") + +# Copied from egomimic/rldb/lance/schema.py (PR #716) so the validator needs no Lance +# install; a unit test checks the two agree once that module is on main. +RECORDING_NAMESPACE = uuid.UUID("6f1d4c3e-2b1a-5e7f-9c8d-0a1b2c3d4e5f") + + +def recording_id(lab: str, vendor_recording_id: str) -> str: + """The Lance recordings.recording_id this episode will get.""" + return str(uuid.uuid5(RECORDING_NAMESPACE, f"{lab}/{vendor_recording_id}")) + + +# Frames every vendor uses by these names in /tf_static and in poses. +WORLD_FRAME = "world" +HEAD_FRAME = "head" + +# ---- Thresholds stated in the plan ---- +MIN_VIDEO_FPS = 30.0 +MIN_IMU_HZ = 200.0 +MIN_HEAD_POSE_HZ = 30.0 +GRAVITY = 9.81 # m/s^2: mean |accel| of a raw IMU at rest +VIDEO_CODECS = ("h264", "h265") +GRIPPER_RANGE = (0.0, 1.0) # 0 = closed, 1 = open + +# ---- PROVISIONAL thresholds (not in the plan) ---- +RATE_TOL = ( + 0.05 # a measured rate may fall this fraction below its minimum (29.97 fps passes) +) +GRAVITY_TOL = 0.5 # m/s^2 allowed around GRAVITY in the stillest window +STILL_WINDOW_S = 1.0 # window used to find the IMU at rest +MAX_GAP_S = 0.5 # largest allowed gap between consecutive messages of one stream +SPAN_TOL_S = 1.0 # how far a stream may start late / end early vs. the episode +METADATA_TIME_TOL_S = 1.0 # start/end-time-unix vs. the data's actual span +UNIT_TOL = 1e-3 # |quaternion| - 1, and R^T R - I for rotation matrices +VIDEO_SAMPLE_FRAMES = 60 # frames decoded per camera to check the stream +MIN_UNIX_NS = 1_420_070_400 * 10**9 # 2015-01-01: anything earlier is not Unix time + +# Must match egomimic.rldb.embodiment.embodiment.EMBODIMENT (lowercased); a unit test +# checks. Listed here so the validator does not import torch. +EMBODIMENTS = ( + "human_right_arm", + "human_left_arm", + "human_bimanual", + "eva_right_arm", + "eva_left_arm", + "eva_bimanual", +) + + +@dataclass(frozen=True) +class Topic: + """One channel an episode may carry. + + `kind` picks the checks the validator runs on it. `schema` is the exact schema name + required, or None for any. `fields` must be non-empty on every message. `side` marks a + per-arm topic, dropped for single-arm embodiments. + """ + + name: str + kind: str + required: bool = True + schema: str | None = None + fields: tuple[str, ...] = () + min_hz: float | None = None + side: str | None = None + + +@dataclass(frozen=True) +class MovingLink: + """A parent -> child transform that changes over time, published on `topic` + instead of /tf_static (e.g. the head moving in the world).""" + + parent: str + child: str + topic: str + + +@dataclass(frozen=True) +class MetaKey: + """One episode-info key. MCAP metadata values are strings; `kind` is what the + string must parse as: "str", "slug" (SLUG), "int", "enum" (one of `values`), + "sha256" (64 lowercase hex digits), "shutter" (SHUTTER) or "hand_joints" + (comma-separated HAND_JOINTS names, each once, including wrist).""" + + name: str + kind: str = "str" + values: tuple[str, ...] = () + human_only: bool = False + + +@dataclass(frozen=True) +class Profile: + name: str + step_clock: str # camera whose frames define the Lance steps rows + topics: tuple[Topic, ...] + moving_links: tuple[MovingLink, ...] + + +_VIDEO = dict( + kind="video", + schema="foxglove.CompressedVideo", + min_hz=MIN_VIDEO_FPS, + fields=("data", "format", "frame_id"), +) +_TF = Topic( + "/tf_static", "tf_static", schema="foxglove.FrameTransforms", fields=("transforms",) +) +_INSTRUCTION = Topic( + "/instruction", "instruction", schema="egoverse.Instruction", fields=("data",) +) +# Optional in either profile. Without /task, the episode-info task / instruction / objects +# keys describe one task spanning the whole recording; with /task, the spans are +# authoritative and those keys only summarize the session (no task row is made from them). +_LABELS = ( + Topic("/task", "task", required=False, schema="egoverse.Task", fields=("name",)), + Topic( + "/subtask-annotation", + "annotation", + required=False, + schema="egoverse.Annotation", + fields=("data",), + ), +) +# Must match egomimic.rldb.lance.schema.ANNOTATION_TYPES (PR #717); empty means "language". +ANNOTATION_TYPES = ("language",) + + +def _per_side(make) -> tuple[Topic, ...]: + return tuple(make(side) for side in ("left", "right")) + + +HUMAN = Profile( + name="human", + step_clock="/main-camera", + topics=( + Topic("/main-camera", **_VIDEO), + Topic("/slam-left-camera", **_VIDEO, required=False), + Topic("/slam-right-camera", **_VIDEO, required=False), + *_per_side(lambda s: Topic(f"/{s}-wrist-camera", **_VIDEO, required=False)), + Topic( + "/imu", + "imu", + schema="egoverse.Imu", + min_hz=MIN_IMU_HZ, + fields=("frame_id", "linear_acceleration", "angular_velocity"), + ), + Topic("/imu-info", "imu_info", schema="egoverse.ImuInfo", fields=("frame_id",)), + Topic( + "/head-pose", + "pose", + schema="foxglove.PoseInFrame", + min_hz=MIN_HEAD_POSE_HZ, + fields=("frame_id", "pose"), + ), + Topic( + "/head-pose-quality", + "tracking_quality", + schema="egoverse.TrackingQuality", + min_hz=MIN_HEAD_POSE_HZ, + ), + *_per_side( + lambda s: Topic( + f"/{s}-hand-keypoints", + "hand_keypoints", + schema="egoverse.HandKeypoints", + fields=("frame_id",), + side=s, + ) + ), + Topic( + "/eye-gaze", + "eye_gaze", + required=False, + schema="egoverse.EyeGaze", + fields=("frame_id",), + ), + _TF, + _INSTRUCTION, + *_LABELS, + ), + moving_links=(MovingLink(WORLD_FRAME, HEAD_FRAME, "/head-pose"),), +) + +ROBOT = Profile( + name="robot", + step_clock="/top-camera", + topics=( + Topic("/top-camera", **_VIDEO), + *_per_side(lambda s: Topic(f"/{s}-wrist-camera", **_VIDEO, side=s)), + *_per_side( + lambda s: Topic( + f"/{s}-arm-state", + "arm", + schema="egoverse.ArmState", + side=s, + fields=( + "frame_id", + "joint_names", + "position", + "velocity", + "torque", + "ee_pose", + ), + ) + ), + *_per_side( + lambda s: Topic( + f"/{s}-arm-action", + "arm", + schema="egoverse.ArmAction", + side=s, + fields=("frame_id", "joint_names", "position", "ee_pose"), + ) + ), + *_per_side( + lambda s: Topic( + f"/{s}-ee-state", "gripper", schema="egoverse.GripperState", side=s + ) + ), + *_per_side( + lambda s: Topic( + f"/{s}-ee-action", "gripper", schema="egoverse.GripperState", side=s + ) + ), + _TF, + _INSTRUCTION, + *_LABELS, + ), + # The end effector moves relative to its arm base; the wrist camera hangs off it. + moving_links=tuple( + MovingLink(f"{s}_arm_base", f"{s}_ee", f"/{s}-arm-state") + for s in ("left", "right") + ), +) + +METADATA_KEYS = ( + MetaKey("lab", "slug"), # data source / Lance partition: aria, eva, mecka, scale + MetaKey("site", "slug"), # organization that collected it: rl2, microagi + MetaKey("operator-id", "sha256"), # SHA-256 of the operator, never a name + MetaKey("task"), + MetaKey("instruction"), + MetaKey("embodiment", "enum", EMBODIMENTS), + MetaKey("rig-name"), + MetaKey("device-serial"), + MetaKey("scene"), + MetaKey("objects"), # comma-separated + MetaKey("start-time-unix", "int"), # ms + MetaKey("end-time-unix", "int"), # ms + # The vendor's own ID for this recording; stable across re-exports and corrections. + MetaKey("vendor-recording-id"), + MetaKey("vendor", "slug"), # who delivered or processed it: acme + MetaKey("spec-version", "enum", SUPPORTED_SPEC_VERSIONS), + MetaKey("license"), # e.g. "CC-BY-4.0"; the Lance recordings.license column + # Where in the exposure frames are stamped. + MetaKey("camera-timestamp-ref", "enum", ("start", "mid", "end"), human_only=True), + MetaKey("shutter", "shutter", human_only=True), # "global" or "rolling:16ms" + # HandKeypoints.positions order, chosen by the vendor, e.g. ",".join(MANO_JOINTS) + MetaKey("hand-joints", "hand_joints", human_only=True), +) + + +def profile_for(embodiment: str) -> Profile: + return HUMAN if embodiment.startswith("human_") else ROBOT + + +def arm_sides(embodiment: str) -> tuple[str, ...]: + """Sides the embodiment uses: ("left", "right") for *_bimanual.""" + if embodiment.endswith("_left_arm"): + return ("left",) + if embodiment.endswith("_right_arm"): + return ("right",) + return ("left", "right") + + +def topics_for(embodiment: str) -> tuple[Topic, ...]: + """The profile's topics, minus per-side topics of an arm the embodiment lacks.""" + sides = arm_sides(embodiment) + return tuple( + t for t in profile_for(embodiment).topics if t.side is None or t.side in sides + ) diff --git a/egomimic/rldb/mcap/validate.py b/egomimic/rldb/mcap/validate.py new file mode 100644 index 0000000000..5256fbe6fb --- /dev/null +++ b/egomimic/rldb/mcap/validate.py @@ -0,0 +1,1214 @@ +"""Check a delivered episode against the vendor MCAP spec (spec.py, SPEC.md). + + python -m egomimic.rldb.mcap.validate episode_/ [episode_/ ...] + +Prints PASS, or every problem found, per episode; exits 1 if any episode fails. The +checks are the plan's six groups, plus the task / label spans the Lance tables need: + + 1. everything there required topics, types, non-empty fields, episode info + 2. video usable codec, starts on a keyframe, decodes, size matches calibration, fps + 3. clocks Unix ns, in order, no big gaps, streams cover the episode, rates + 4. calibration every camera has intrinsics and a path to the world frame + 5. imu raw (gravity still in it) and fast enough, noise specs present + 6. poses no NaNs, unit rotations, gripper in [0, 1], lost hands flagged not zeroed + 7. tasks & labels /task and /subtask-annotation spans are ordered, inside the data, + each label inside one task (the rules Lance's annotation review + applies, egomimic/rldb/lance/annotations.py in PR #717) + +The file is read once, in file order. Video is only decoded for its first +VIDEO_SAMPLE_FRAMES messages, each of which must decode to exactly one frame, so that +every frame has its own capture time (Lance needs one steps row per frame). annotation.mcap, when present, +is read too and may only hold the label topics. + +MCAP reading and packet-by-packet video decoding follow PR #606's +egomimic/scripts/abc_process/abc_to_zarr.py. +""" + +import argparse +import math +import re +import sys +from collections import Counter, defaultdict +from dataclasses import dataclass, field +from pathlib import Path + +import numpy as np + +from egomimic.rldb.mcap import spec + +GROUPS = { + "present": "everything there", + "video": "video usable", + "clocks": "clocks", + "calibration": "calibration", + "imu": "imu", + "poses": "poses", + "labels": "tasks & labels", +} +# Topics whose messages form a time series; the rest are published once or rarely. +STREAM_KINDS = { + "video", + "imu", + "pose", + "tracking_quality", + "hand_keypoints", + "arm", + "gripper", + "eye_gaze", +} + + +def _reach(edges: dict, start: str) -> set[str]: + reached, todo = {start}, [start] + while todo: + for nxt in edges[todo.pop()] - reached: + reached.add(nxt) + todo.append(nxt) + return reached + + +def frames_in_message(data: bytes, codec: str) -> int: + """Pictures (access units) in one video message; the spec wants exactly 1, so every + frame has its own capture time. Counted with the parser: the decoder silently drops + the extra frames of a packed message. Shared with the MCAP -> Lance converter.""" + import av + + parser = av.CodecContext.create("hevc" if codec == "h265" else "h264", "r") + with av.logging.Capture(): # lone P-frames log "non-existing PPS" + return len(parser.parse(data)) + len(parser.parse(None)) + + +# Topics allowed in annotation.mcap. +LABEL_TOPICS = {"/task", "/subtask-annotation"} +# Annex B NAL unit types that start a decodable picture (IDR / IRAP). +KEYFRAME_NALS = {"h264": {5}, "h265": set(range(16, 22))} + + +@dataclass(frozen=True) +class Problem: + group: str + topic: str | None + message: str + + def __str__(self) -> str: + where = f"{self.topic}: " if self.topic else "" + return f"[{GROUPS[self.group]}] {where}{self.message}" + + +@dataclass +class _Stream: + """Everything one pass keeps about one topic.""" + + schema: str + encoding: str + times: list[int] = field(default_factory=list) + msgs: list = field(default_factory=list) # decoded; video keeps only the sample + empty: Counter = field( + default_factory=Counter + ) # field -> messages where it is empty + missing_fields: set = field(default_factory=set) # fields the message type lacks + undecodable: bool = False + + @property + def n(self) -> int: + return len(self.times) + + def span_s(self) -> float: + return (max(self.times) - min(self.times)) / 1e9 if self.times else 0.0 + + def rate_hz(self) -> float: + span = self.span_s() + return (self.n - 1) / span if span > 0 else 0.0 + + +@dataclass +class _Episode: + path: Path + metadata: dict[str, dict[str, str]] # record name -> its key/values + streams: dict[str, _Stream] + + @property + def info(self) -> dict[str, str]: + return self.metadata.get(spec.METADATA_RECORD, {}) + + +# ---------------------------------------------------------------- reading + + +def _is_empty(msg, name: str) -> bool: + value = getattr(msg, name) + if isinstance(value, (str, bytes)) or hasattr(value, "__len__"): + return len(value) == 0 + try: + return not msg.HasField(name) # sub-message + except ValueError: + return False # proto3 scalar: zero is a legitimate value + + +def _read(path: Path) -> _Episode: + from mcap.reader import make_reader + from mcap_protobuf.decoder import DecoderFactory + + fields_of = {t.name: t.fields for p in (spec.HUMAN, spec.ROBOT) for t in p.topics} + video_topics = { + t.name for p in (spec.HUMAN, spec.ROBOT) for t in p.topics if t.kind == "video" + } + factory = DecoderFactory() + decoders: dict[int, object] = {} + streams: dict[str, _Stream] = {} + + with open(path, "rb") as f: + reader = make_reader(f) + metadata = {r.name: dict(r.metadata) for r in reader.iter_metadata()} + for schema, channel, message in reader.iter_messages(log_time_order=False): + topic = channel.topic + s = streams.get(topic) + if s is None: + s = streams[topic] = _Stream( + schema.name if schema else "", channel.message_encoding + ) + s.times.append(message.log_time) + if channel.id not in decoders: + decoders[channel.id] = factory.decoder_for( + channel.message_encoding, schema + ) + decode = decoders[channel.id] + if decode is None: + s.undecodable = True + continue + is_video = topic in video_topics or topic.endswith(spec.CAMERA_SUFFIX) + if is_video and len(s.msgs) >= spec.VIDEO_SAMPLE_FRAMES: + continue # video is checked on its sample; decoding all of it is costly + msg = decode(message.data) + for name in fields_of.get(topic, ()): + if not hasattr(msg, name): + s.missing_fields.add(name) + elif _is_empty(msg, name): + s.empty[name] += 1 + s.msgs.append(msg) + return _Episode(path, metadata, streams) + + +# ---------------------------------------------------------------- 1. everything there + + +def _embodiment(ep: _Episode) -> str: + """Declared embodiment, or a guess from the topics when it is missing/invalid.""" + declared = ep.info.get("embodiment", "") + if declared in spec.EMBODIMENTS: + return declared + return "human_bimanual" if "/main-camera" in ep.streams else "eva_bimanual" + + +def _int(value: str) -> int | None: + """The integer an episode-info value spells, or None (int() alone takes "²", " 5").""" + return int(value) if re.fullmatch(r"-?[0-9]+", value) else None + + +def _hand_joints_problem(value: str) -> str: + names = value.split(",") + unknown = [n for n in names if n not in spec.HAND_JOINTS] + if unknown: + return f"unknown joints {unknown}; use names from {list(spec.HAND_JOINTS)}" + if len(set(names)) != len(names): + return "lists a joint twice" + if "wrist" not in names: + return "must include wrist" + return "" + + +def _hand_joint_count(ep: _Episode) -> int: + """Joints per hand the episode declares; MANO's 21 when the key is unusable.""" + value = ep.info.get("hand-joints", "") + if value and not _hand_joints_problem(value): + return len(value.split(",")) + return len(spec.MANO_JOINTS) + + +def check_present(ep: _Episode, embodiment: str) -> list[Problem]: + out = [] + expected = spec.topics_for(embodiment) + for t in expected: + s = ep.streams.get(t.name) + if s is None: + if t.required: + out.append(Problem("present", t.name, "missing required topic")) + continue + if t.schema and s.schema != t.schema: + out.append( + Problem( + "present", t.name, f"type is {s.schema!r}, expected {t.schema!r}" + ) + ) + if s.encoding != "protobuf" or s.undecodable: + out.append( + Problem( + "present", + t.name, + f"cannot decode {s.encoding!r} messages; the spec uses protobuf", + ) + ) + for name in sorted(s.missing_fields): + out.append( + Problem("present", t.name, f"message type has no field {name!r}") + ) + for name, count in sorted(s.empty.items()): + out.append( + Problem("present", t.name, f"{name!r} empty on {count}/{s.n} messages") + ) + + # The embodiment must match the topics: no topics of the other profile or an unused arm. + allowed = {t.name for t in expected} + for p in (spec.HUMAN, spec.ROBOT): + for t in p.topics: + if t.name in ep.streams and t.name not in allowed: + out.append( + Problem( + "present", + t.name, + f"topic does not fit embodiment {embodiment!r}", + ) + ) + + human = spec.profile_for(embodiment) is spec.HUMAN + if spec.METADATA_RECORD not in ep.metadata: + found = ", ".join(sorted(ep.metadata)) or "none" + out.append( + Problem( + "present", + None, + f"no {spec.METADATA_RECORD!r} metadata record (found: {found})", + ) + ) + return out + for key in spec.METADATA_KEYS: + if key.human_only and not human: + continue + value = ep.info.get(key.name) + if value is None or value == "": + out.append(Problem("present", None, f"episode info {key.name!r} missing")) + elif key.kind == "sha256" and not re.fullmatch(r"[0-9a-f]{64}", value): + out.append( + Problem( + "present", + None, + f"episode info {key.name!r} is not a SHA-256 hex digest " + "(hash names and emails)", + ) + ) + elif key.kind == "shutter" and not spec.SHUTTER.fullmatch(value): + out.append( + Problem( + "present", + None, + f"episode info {key.name!r} = {value!r}, expected 'global' or " + "'rolling:ms'", + ) + ) + elif key.kind == "slug" and not spec.SLUG.fullmatch(value): + out.append( + Problem( + "present", + None, + f"episode info {key.name!r} = {value!r} is not a lowercase slug " + "(a-z, 0-9, words joined by _ or -)", + ) + ) + elif key.kind == "hand_joints" and (bad := _hand_joints_problem(value)): + out.append(Problem("present", None, f"episode info {key.name!r}: {bad}")) + elif key.kind == "int" and _int(value) is None: + out.append( + Problem( + "present", + None, + f"episode info {key.name!r} = {value!r} is not an integer", + ) + ) + elif key.kind == "enum" and value not in key.values: + out.append( + Problem( + "present", + None, + f"episode info {key.name!r} = {value!r}, expected one of {list(key.values)}", + ) + ) + return out + + +# ---------------------------------------------------------------- 2. video usable + + +def _codec(fmt: str) -> str | None: + fmt = fmt.lower() + return fmt if fmt in spec.VIDEO_CODECS else None + + +def _nal_types(data: bytes, codec: str) -> set[int]: + types, i = set(), 0 + while (j := data.find(b"\x00\x00\x01", i)) >= 0 and j + 3 < len(data): + b = data[j + 3] + types.add(b & 0x1F if codec == "h264" else (b >> 1) & 0x3F) + i = j + 3 + return types + + +def _calibration(ep: _Episode, camera: str): + s = ep.streams.get(camera + spec.CALIBRATION_SUFFIX) + return s.msgs[0] if s and s.msgs else None + + +def check_video(ep: _Episode) -> list[Problem]: + import av + + out = [] + for topic, s in sorted(ep.streams.items()): + if not topic.endswith(spec.CAMERA_SUFFIX) or not s.msgs: + continue + fmt = str(getattr(s.msgs[0], "format", "")) + codec = _codec(fmt) + if codec is None: + out.append( + Problem( + "video", + topic, + f"format {fmt!r}, expected one of {list(spec.VIDEO_CODECS)}", + ) + ) + continue + keyframes = [ + i + for i, msg in enumerate(s.msgs) + if _nal_types(bytes(msg.data), codec) & KEYFRAME_NALS[codec] + ] + if not keyframes: + out.append( + Problem( + "video", + topic, + f"no keyframe (Annex B IDR/IRAP) in the first {len(s.msgs)} messages", + ) + ) + continue + if keyframes[0] != 0: + out.append( + Problem( + "video", topic, "first message is not a keyframe (Annex B IDR/IRAP)" + ) + ) + + # Decode from the first keyframe, so a bad start is reported once, above. + sample = s.msgs[keyframes[0] :] + + packed = sum(frames_in_message(bytes(m.data), codec) != 1 for m in sample) + if packed: + out.append( + Problem( + "video", + topic, + f"{packed}/{len(sample)} sampled messages do not hold exactly one " + "frame; each message must be one frame at its capture time", + ) + ) + ctx = av.CodecContext.create("hevc" if codec == "h265" else "h264", "r") + frames, size = 0, None + try: + for msg in sample: + for frame in ctx.decode(av.Packet(bytes(msg.data))): + frames += 1 + size = size or (frame.width, frame.height) + for frame in ctx.decode(None): + frames += 1 + size = size or (frame.width, frame.height) + except av.error.FFmpegError as e: + out.append( + Problem( + "video", + topic, + f"decode error in the first {len(sample)} messages: {e}", + ) + ) + continue + if frames == 0: + out.append( + Problem( + "video", + topic, + f"no frames decoded from the first {len(sample)} messages", + ) + ) + continue + + calib = _calibration(ep, topic) + if calib is not None and size != (int(calib.width), int(calib.height)): + out.append( + Problem( + "video", + topic, + f"frames are {size[0]}x{size[1]}, calibration says {calib.width}x{calib.height}", + ) + ) + + fps = s.rate_hz() # meaningless for packed messages, reported above + if not packed and fps < spec.MIN_VIDEO_FPS * (1 - spec.RATE_TOL): + out.append( + Problem( + "video", topic, f"{fps:.1f} fps, need >= {spec.MIN_VIDEO_FPS:g}" + ) + ) + return out + + +# ---------------------------------------------------------------- 3. clocks + + +def _time_series(ep: _Episode, embodiment: str) -> dict[str, _Stream]: + """Cameras and the spec's time-series topics: what defines the recording's span.""" + expected = {t.name: t for t in spec.topics_for(embodiment)} + return { + topic: s + for topic, s in ep.streams.items() + if s.times + and ( + topic.endswith(spec.CAMERA_SUFFIX) + or (topic in expected and expected[topic].kind in STREAM_KINDS) + ) + } + + +def _span(streams: dict[str, _Stream]) -> tuple[int, int]: + return ( + min(min(s.times) for s in streams.values()), + max(max(s.times) for s in streams.values()), + ) + + +def check_clocks(ep: _Episode, embodiment: str) -> list[Problem]: + out = [] + expected = {t.name: t for t in spec.topics_for(embodiment)} + streams = _time_series(ep, embodiment) + for topic, s in sorted(ep.streams.items()): + if s.times and min(s.times) < spec.MIN_UNIX_NS: + out.append( + Problem( + "clocks", topic, "timestamps are before 2015: not absolute Unix ns" + ) + ) + for topic, s in sorted(streams.items()): + t = np.asarray(s.times, dtype=np.int64) + back = int((np.diff(t) < 0).sum()) + if back: + out.append( + Problem("clocks", topic, f"timestamps go backwards {back} times") + ) + same = int((np.diff(t) == 0).sum()) + if same and topic.endswith(spec.CAMERA_SUFFIX): + # Every frame needs its own time: Lance steps timestamps strictly increase. + out.append(Problem("clocks", topic, f"{same} frames repeat a timestamp")) + if len(t) > 1: + gap = float(np.diff(np.sort(t)).max()) / 1e9 + if gap > spec.MAX_GAP_S: + out.append( + Problem( + "clocks", topic, f"{gap:.2f} s gap, max {spec.MAX_GAP_S:g} s" + ) + ) + # Video rates are checked from decoded frames; IMU rate in its own group. + t_spec = expected.get(topic) + if t_spec and t_spec.min_hz and t_spec.kind not in ("video", "imu"): + if s.rate_hz() < t_spec.min_hz * (1 - spec.RATE_TOL): + out.append( + Problem( + "clocks", + topic, + f"{s.rate_hz():.1f} Hz, need >= {t_spec.min_hz:g}", + ) + ) + + if not streams: + return out + start, end = _span(streams) + for topic, s in sorted(streams.items()): + late, early = (min(s.times) - start) / 1e9, (end - max(s.times)) / 1e9 + if late > spec.SPAN_TOL_S or early > spec.SPAN_TOL_S: + out.append( + Problem( + "clocks", + topic, + f"covers only part of the episode (starts {late:.1f} s late, ends {early:.1f} s early)", + ) + ) + + for key, data_ns in (("start-time-unix", start), ("end-time-unix", end)): + value = ep.info.get(key, "") + if _int(value) is not None: + off = abs(_int(value) * 1e6 - data_ns) / 1e9 + if off > spec.METADATA_TIME_TOL_S: + out.append( + Problem( + "clocks", + None, + f"episode info {key!r} is {off:.1f} s from the data", + ) + ) + return out + + +# ---------------------------------------------------------------- 4. calibration + + +def _quat_ok(q) -> bool: + v = [q.x, q.y, q.z, q.w] + return ( + all(math.isfinite(c) for c in v) + and abs(math.sqrt(sum(c * c for c in v)) - 1) <= spec.UNIT_TOL + ) + + +def check_calibration(ep: _Episode, embodiment: str) -> list[Problem]: + out = [] + cameras = sorted(t for t in ep.streams if t.endswith(spec.CAMERA_SUFFIX)) + for cam in cameras: + calib = _calibration(ep, cam) + if calib is None: + out.append( + Problem( + "calibration", + cam, + f"no {cam + spec.CALIBRATION_SUFFIX} calibration", + ) + ) + continue + video = ep.streams.get(cam) + if video and video.msgs and hasattr(video.msgs[0], "frame_id"): + named = {str(m.frame_id) for m in video.msgs} + if named != {str(calib.frame_id)}: + out.append( + Problem( + "calibration", + cam, + f"video frames name {sorted(named)} but its calibration names " + f"{calib.frame_id!r}: they must be the camera's one frame", + ) + ) + info = ep.streams[cam + spec.CALIBRATION_SUFFIX] + if info.schema != "foxglove.CameraCalibration" or info.n != 1: + out.append( + Problem( + "calibration", + cam + spec.CALIBRATION_SUFFIX, + f"expected one foxglove.CameraCalibration, got {info.n} " + f"{info.schema or 'untyped'} message(s)", + ) + ) + K = list(getattr(calib, "K", [])) + D = list(getattr(calib, "D", [])) + if not all(math.isfinite(v) for v in K + D): + out.append(Problem("calibration", cam, "K or D has NaN / Inf")) + elif len(K) == 9 and not ( + 0 <= K[2] <= calib.width and 0 <= K[5] <= calib.height + ): + out.append( + Problem( + "calibration", + cam, + f"principal point ({K[2]:g}, {K[5]:g}) is outside the " + f"{calib.width}x{calib.height} image", + ) + ) + if len(K) != 9 or K[0] <= 0 or K[4] <= 0 or not calib.width or not calib.height: + out.append( + Problem( + "calibration", + cam, + "calibration has no valid K (fx, fy > 0) or size", + ) + ) + model, n_d = str(calib.distortion_model), len(calib.D) + if model == "" and n_d == 0: + pass # pinhole + elif model not in spec.DISTORTION_MODELS: + out.append( + Problem( + "calibration", + cam, + f"distortion_model {model!r}, expected one of " + f"{list(spec.DISTORTION_MODELS)}", + ) + ) + elif n_d != spec.DISTORTION_MODELS[model]: + out.append( + Problem( + "calibration", + cam, + f"{model} needs {spec.DISTORTION_MODELS[model]} D coefficients, " + f"got {n_d}", + ) + ) + + extra = ep.streams.get(cam + spec.CAMERA_EXTRA_SUFFIX) + if extra and extra.msgs: + m = extra.msgs[0] + if m.upright_rotation_cw_deg not in spec.UPRIGHT_ROTATIONS: + out.append( + Problem( + "calibration", + cam, + f"upright_rotation_cw_deg {m.upright_rotation_cw_deg}, expected " + f"one of {list(spec.UPRIGHT_ROTATIONS)}", + ) + ) + if not (math.isfinite(m.valid_radius) and m.valid_radius >= 0): + out.append( + Problem( + "calibration", cam, f"valid_radius {m.valid_radius} is not >= 0" + ) + ) + if m.frame_id != str(getattr(calib, "frame_id", m.frame_id)): + out.append( + Problem( + "calibration", + cam + spec.CAMERA_EXTRA_SUFFIX, + f"frame_id {m.frame_id!r} differs from the camera's -info", + ) + ) + + # Frame graph: /tf_static edges plus the moving links whose topic is present. + edges = defaultdict(set) + static = defaultdict(set) # /tf_static alone: what is rigidly mounted to what + tf = ep.streams.get("/tf_static") + bad_rot = bad_tr = 0 + for msg in tf.msgs if tf else (): + for x in msg.transforms: + for g in (edges, static): + g[x.parent_frame_id].add(x.child_frame_id) + g[x.child_frame_id].add(x.parent_frame_id) + bad_rot += not _quat_ok(x.rotation) + t = x.translation + bad_tr += not all(math.isfinite(v) for v in (t.x, t.y, t.z)) + if bad_rot: + out.append( + Problem( + "calibration", + "/tf_static", + f"{bad_rot} transforms have a non-unit rotation", + ) + ) + if bad_tr: + out.append( + Problem( + "calibration", + "/tf_static", + f"{bad_tr} transforms have a NaN / Inf translation", + ) + ) + sides = spec.arm_sides(embodiment) + for link in spec.profile_for(embodiment).moving_links: + if link.topic in ep.streams and ( + not link.parent.endswith("_arm_base") or link.parent.split("_")[0] in sides + ): + edges[link.parent].add(link.child) + edges[link.child].add(link.parent) + reached = _reach(edges, spec.WORLD_FRAME) + + # A head-worn rig's steps camera rides on the head: its frame must be rigidly + # mounted to the head (a /tf_static path), which is how the steps are placed. + profile = spec.profile_for(embodiment) + if any(link.child == spec.HEAD_FRAME for link in profile.moving_links): + clock = _calibration(ep, profile.step_clock) + fid = str(clock.frame_id) if clock is not None else "" + # (An unplaced frame is already reported below as having no path to world.) + if fid in reached and fid not in _reach(static, spec.HEAD_FRAME): + out.append( + Problem( + "calibration", + profile.step_clock, + f"frame {clock.frame_id!r} has no fixed path to " + f"{spec.HEAD_FRAME!r} via /tf_static: the steps camera must be " + "mounted on the head", + ) + ) + + # Cameras and the IMU must name their frame; every frame any topic names (hands, + # wrist poses, gaze, ...) must be placed in the world. + must_name = set(cameras) | {"/imu"} + for topic, s in sorted(ep.streams.items()): + skip = topic == "/tf_static" or topic.endswith( + (spec.CALIBRATION_SUFFIX, spec.CAMERA_EXTRA_SUFFIX) + ) + if skip or not s.msgs or not hasattr(s.msgs[0], "frame_id"): + continue # a camera's -info / -info-extra share its frame, checked above + named = {str(m.frame_id) for m in s.msgs} + if "" in named and topic in must_name: + out.append( + Problem("calibration", topic, "no frame_id, so it cannot be placed") + ) + for frame in sorted(named - reached - {""}): + out.append( + Problem( + "calibration", + topic, + f"frame {frame!r} has no path to {spec.WORLD_FRAME!r} via /tf_static", + ) + ) + head = ep.streams.get("/head-pose") + if ( + head + and head.msgs + and {str(m.frame_id) for m in head.msgs} != {spec.WORLD_FRAME} + ): + out.append( + Problem( + "calibration", + "/head-pose", + f"poses must be in frame {spec.WORLD_FRAME!r} (T_world_head)", + ) + ) + return out + + +# ---------------------------------------------------------------- 5. imu + + +def check_imu(ep: _Episode) -> list[Problem]: + out = [] + s = ep.streams.get("/imu") + if s and s.msgs: + if s.rate_hz() < spec.MIN_IMU_HZ * (1 - spec.RATE_TOL): + out.append( + Problem( + "imu", "/imu", f"{s.rate_hz():.0f} Hz, need >= {spec.MIN_IMU_HZ:g}" + ) + ) + acc = np.array( + [ + [ + m.linear_acceleration.x, + m.linear_acceleration.y, + m.linear_acceleration.z, + ] + for m in s.msgs + ] + ) + gyr = np.array( + [ + [m.angular_velocity.x, m.angular_velocity.y, m.angular_velocity.z] + for m in s.msgs + ] + ) + if not (np.isfinite(acc).all() and np.isfinite(gyr).all()): + out.append(Problem("imu", "/imu", "NaN/Inf samples")) + else: + # Stillest window = lowest mean rotation rate; a raw IMU there reads ~9.81 m/s^2. + w = max(1, min(len(acc), round(s.rate_hz() * spec.STILL_WINDOW_S))) + gyro_mag = np.convolve( + np.linalg.norm(gyr, axis=1), np.ones(w) / w, mode="valid" + ) + i = int(np.argmin(gyro_mag)) + still = float(np.linalg.norm(acc[i : i + w], axis=1).mean()) + if abs(still - spec.GRAVITY) > spec.GRAVITY_TOL: + out.append( + Problem( + "imu", + "/imu", + f"mean |accel| at rest = {still:.2f} m/s^2, expected ~{spec.GRAVITY} (gravity removed, or wrong units?)", + ) + ) + info = ep.streams.get("/imu-info") + if info and info.msgs: + names = ( + "accel_noise_density", + "accel_random_walk", + "gyro_noise_density", + "gyro_random_walk", + ) + bad = [n for n in names if not getattr(info.msgs[0], n, 0) > 0] + if bad: + out.append( + Problem( + "imu", "/imu-info", f"noise values must be > 0: {', '.join(bad)}" + ) + ) + return out + + +# ---------------------------------------------------------------- 6. poses + + +def _pose_ok(pose) -> bool: + p = pose.position + return _quat_ok(pose.orientation) and all(math.isfinite(v) for v in (p.x, p.y, p.z)) + + +def check_poses(ep: _Episode, embodiment: str) -> list[Problem]: + out = [] + for t in spec.topics_for(embodiment): + s = ep.streams.get(t.name) + if not s or not s.msgs: + continue + if t.kind == "pose": + bad = sum(not _pose_ok(m.pose) for m in s.msgs) + if bad: + out.append( + Problem( + "poses", + t.name, + f"{bad}/{s.n} poses have NaNs or a non-unit rotation", + ) + ) + elif t.kind == "hand_keypoints": + joints = _hand_joint_count(ep) + shape = sum(len(m.positions) not in (0, 3 * joints) for m in s.msgs) + if shape: + out.append( + Problem( + "poses", + t.name, + f"{shape}/{s.n} messages have neither {3 * joints} positions " + f"({joints} hand-joints x xyz) nor none", + ) + ) + empty = sum( + m.valid and not m.positions and not m.HasField("wrist_pose") + for m in s.msgs + ) + if empty: + out.append( + Problem( + "poses", + t.name, + f"{empty} valid messages have neither positions nor wrist_pose", + ) + ) + conf = sum( + not 0 <= m.confidence <= 1 + or len(m.joint_confidence) not in (0, joints) + or not all(0 <= c <= 1 for c in m.joint_confidence) + for m in s.msgs + ) + if conf: + out.append( + Problem( + "poses", + t.name, + f"{conf}/{s.n} messages have a confidence outside [0, 1] or " + f"joint_confidence not {joints} values", + ) + ) + wrist = sum( + m.valid and m.HasField("wrist_pose") and not _pose_ok(m.wrist_pose) + for m in s.msgs + ) + if wrist: + out.append( + Problem( + "poses", + t.name, + f"{wrist} valid wrist poses have NaNs or a non-unit rotation", + ) + ) + valid = [ + np.asarray(m.positions) + for m in s.msgs + if m.valid and len(m.positions) == 3 * joints + ] + nan = sum(not np.isfinite(p).all() for p in valid) + zero = sum(bool(np.isfinite(p).all() and not p.any()) for p in valid) + if nan: + out.append( + Problem( + "poses", t.name, f"{nan} valid messages have NaN/Inf positions" + ) + ) + if zero: + out.append( + Problem( + "poses", + t.name, + f"{zero} messages are all-zero with valid=true; lost tracking must be valid=false", + ) + ) + elif t.kind == "eye_gaze": + bad = sum( + m.valid and not all(math.isfinite(v) for v in (m.yaw, m.pitch, m.depth)) + for m in s.msgs + ) + if bad: + out.append( + Problem("poses", t.name, f"{bad} valid messages have NaN/Inf") + ) + elif t.kind == "arm": + names = sum( + len(m.joint_names) != len(m.position) + or (hasattr(m, "velocity") and len(m.velocity) != len(m.position)) + or (hasattr(m, "torque") and len(m.torque) != len(m.position)) + for m in s.msgs + ) + if names: + out.append( + Problem( + "poses", + t.name, + f"{names}/{s.n} messages' joint arrays do not match joint_names", + ) + ) + bad = 0 + for m in s.msgs: + pose = np.asarray(m.ee_pose, dtype=np.float64) + if ( + len(pose) != 16 + or not np.isfinite(pose).all() + or not np.isfinite(np.asarray(m.position)).all() + ): + bad += 1 + continue + T = pose.reshape(4, 4) + R = T[:3, :3] + if ( + np.abs(R.T @ R - np.eye(3)).max() > spec.UNIT_TOL + or abs(np.linalg.det(R) - 1) > spec.UNIT_TOL + or np.abs(T[3] - [0, 0, 0, 1]).max() > spec.UNIT_TOL + ): + bad += 1 + if bad: + out.append( + Problem( + "poses", + t.name, + f"{bad}/{s.n} poses are not a finite rigid 4x4 transform", + ) + ) + elif t.kind == "gripper": + lo, hi = spec.GRIPPER_RANGE + vals = np.array([m.position for m in s.msgs], dtype=np.float64) + bad = int((~np.isfinite(vals) | (vals < lo) | (vals > hi)).sum()) + if bad: + out.append( + Problem( + "poses", + t.name, + f"{bad}/{s.n} gripper values outside [{lo:g}, {hi:g}]", + ) + ) + return out + + +# ---------------------------------------------------------------- 7. tasks & labels + + +def _spans(s: _Stream | None) -> list[tuple[int, int]]: + """(start, end) per message; end 0 (unset) gives end == start: a point event for a + label (one frame wide in Lance), an error for a task.""" + if not s or not s.msgs: + return [] + return [ + (t, int(getattr(m, "end_time_unix_ns", 0)) or t) + for t, m in zip(s.times, s.msgs) + ] + + +def check_labels(ep: _Episode, embodiment: str) -> list[Problem]: + out = [] + streams = _time_series(ep, embodiment) + if not streams: + return out + start, end = _span(streams) + tol = spec.SPAN_TOL_S * 1e9 + tasks = _spans(ep.streams.get("/task")) + labels = _spans(ep.streams.get("/subtask-annotation")) + # A point label becomes one steps-camera frame wide in Lance (as the converter writes + # it), and Lance's annotation review wants the whole span inside a task. + cam = ep.streams.get(spec.profile_for(embodiment).step_clock) + cam_t = np.sort(np.asarray(cam.times if cam else [], dtype=np.int64)) + frame = float(np.median(np.diff(cam_t))) if len(cam_t) > 1 else 0.0 + wide = [(b, e if e > b else b + frame) for b, e in labels] + for topic, spans, point_ok in ( + ("/task", tasks, False), + ("/subtask-annotation", labels, True), + ): + backwards = sum(e < b or (e == b and not point_ok) for b, e in spans) + outside = sum(b < start - tol or e > end + tol for b, e in spans) + if backwards: + out.append( + Problem( + "labels", topic, f"{backwards} spans do not end after they start" + ) + ) + if outside: + out.append( + Problem( + "labels", + topic, + f"{outside}/{len(spans)} spans fall outside the recorded data", + ) + ) + if tasks: + orphans = sum(not any(tb <= b and e <= te for tb, te in tasks) for b, e in wide) + if orphans: + out.append( + Problem( + "labels", + "/subtask-annotation", + f"{orphans}/{len(labels)} labels do not lie inside one /task span", + ) + ) + else: + # The task made from the episode info: [first frame, last frame + one frame). + if len(cam_t) and labels: + first, end = int(cam_t[0]), int(cam_t[-1]) + frame + beyond = sum(b < first or e > end for b, e in wide) + if beyond: + out.append( + Problem( + "labels", + "/subtask-annotation", + f"{beyond}/{len(labels)} labels fall outside " + f"{spec.profile_for(embodiment).step_clock}'s frames (no /task)", + ) + ) + s = ep.streams.get("/subtask-annotation") + types = ( + {m.annotation_type for m in s.msgs if hasattr(m, "annotation_type")} + if s + else set() + ) + bad = sorted(types - {"", *spec.ANNOTATION_TYPES}) + if bad: + out.append( + Problem( + "labels", + "/subtask-annotation", + f"annotation_type {bad}, expected one of {list(spec.ANNOTATION_TYPES)}", + ) + ) + return out + + +# ---------------------------------------------------------------- entry points + + +@dataclass +class Result: + embodiment: str # declared, or guessed from the topics; "" if unreadable + recording_id: str # the Lance recording_id it will get; "" without lab / vendor id + problems: list[Problem] + + @property + def ok(self) -> bool: + return not self.problems + + +def validate_episode(episode: Path) -> Result: + """Validate one episode directory (or its episode.mcap).""" + episode = Path(episode) + path = episode / spec.EPISODE_FILE if episode.is_dir() else episode + if not path.exists(): + return Result( + "", "", [Problem("present", None, f"no {spec.EPISODE_FILE} in {episode}")] + ) + try: + ep = _read(path) + except Exception as e: # truncated or not an MCAP at all + return Result( + "", + "", + [ + Problem( + "present", None, f"cannot read {path.name}: {type(e).__name__}: {e}" + ) + ], + ) + problems = [] + annotation = path.with_name(spec.ANNOTATION_FILE) + if path.name == spec.EPISODE_FILE and annotation.exists(): + try: + extra = _read(annotation) + except Exception as e: + extra = None + problems.append( + Problem( + "present", + None, + f"cannot read {annotation.name}: {type(e).__name__}: {e}", + ) + ) + if extra and extra.metadata: + problems.append( + Problem( + "present", + None, + f"{annotation.name} carries metadata records {sorted(extra.metadata)}; " + f"episode info belongs in {path.name} only", + ) + ) + for topic, s in sorted(extra.streams.items()) if extra else (): + if topic not in LABEL_TOPICS: + problems.append( + Problem( + "present", + topic, + f"only {sorted(LABEL_TOPICS)} belong in {annotation.name}", + ) + ) + elif topic in ep.streams: + problems.append( + Problem( + "present", topic, f"in both {path.name} and {annotation.name}" + ) + ) + else: + ep.streams[topic] = s + embodiment = _embodiment(ep) + for group, check, args in ( + ("present", check_present, (ep, embodiment)), + ("video", check_video, (ep,)), + ("clocks", check_clocks, (ep, embodiment)), + ("calibration", check_calibration, (ep, embodiment)), + ("imu", check_imu, (ep,)), + ("poses", check_poses, (ep, embodiment)), + ("labels", check_labels, (ep, embodiment)), + ): + try: + problems += check(*args) + except Exception as e: # a topic of the wrong type the group's checks assume + problems.append( + Problem( + group, + None, + f"could not check: {type(e).__name__}: {e} (see 'type is' problems)", + ) + ) + lab, vid = ep.info.get("lab", ""), ep.info.get("vendor-recording-id", "") + rid = spec.recording_id(lab, vid) if lab and vid else "" + return Result(embodiment, rid, problems) + + +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + p.add_argument( + "episodes", + nargs="+", + type=Path, + help="episode_/ dirs or episode.mcap files", + ) + args = p.parse_args(argv) + failed = 0 + for episode in args.episodes: + r = validate_episode(episode) + label = ( + f"{episode} (spec {spec.SPEC_VERSION}, embodiment {r.embodiment or '?'}, " + f"recording_id {r.recording_id or '?'})" + ) + if r.problems: + failed += 1 + print(f"FAIL {label}: {len(r.problems)} problem(s)") + for problem in r.problems: + print(f" {problem}") + else: + print(f"PASS {label}") + if len(args.episodes) > 1: + print(f"\n{len(args.episodes) - failed}/{len(args.episodes)} passed") + return 1 if failed else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/pyproject.toml b/pyproject.toml index 9d1213ec3b..c38e6ab7c0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -65,6 +65,10 @@ dependencies = [ "openai", "scikit-learn>=1.8.0", "scipy", + # Vendor MCAP deliveries (egomimic/rldb/mcap) + "mcap==1.5.0", + "mcap-protobuf-support==0.5.4", + "foxglove-schemas-protobuf==0.4.0", ] # Not declared: `openpi` (egomimic/algo/pi.py). The fork at external/openpi pins @@ -84,7 +88,7 @@ where = ["."] include = ["egomimic*"] [tool.setuptools.package-data] -egomimic = ["hydra_configs/**/*.yaml", "resources/*"] +egomimic = ["hydra_configs/**/*.yaml", "resources/*", "rldb/mcap/proto/*.proto"] [tool.pytest.ini_options] testpaths = ["tests"] diff --git a/tests/fixtures/mcap_episodes.py b/tests/fixtures/mcap_episodes.py new file mode 100644 index 0000000000..e092aa7119 --- /dev/null +++ b/tests/fixtures/mcap_episodes.py @@ -0,0 +1,551 @@ +"""Tiny synthetic vendor MCAP episodes (spec 0.1), valid unless a breakage is asked for. + +`write_mcap_episode` writes `episode_/episode.mcap` with every topic the spec +requires for the embodiment: ~2 s of 64x48 H.264 video, 200 Hz IMU, 30 Hz poses, etc. +It also writes the optional label topics (two /task spans with a /subtask-annotation +each, in annotation.mcap by default) and, for humans, /eye-gaze. +Each keyword-only switch breaks exactly one thing, so a test can assert the validator +reports that problem and nothing else. +""" + +from __future__ import annotations + +from collections.abc import Callable +from fractions import Fraction +from functools import lru_cache +from pathlib import Path + +import numpy as np +from foxglove_schemas_protobuf.CameraCalibration_pb2 import CameraCalibration +from foxglove_schemas_protobuf.CompressedVideo_pb2 import CompressedVideo +from foxglove_schemas_protobuf.FrameTransforms_pb2 import FrameTransforms +from foxglove_schemas_protobuf.PoseInFrame_pb2 import PoseInFrame +from mcap_protobuf.writer import Writer + +from egomimic.rldb.mcap import schemas, spec + +T0 = 1_760_000_000 * 10**9 # 2025-10-09, absolute Unix ns +DURATION_S = 2 +WIDTH, HEIGHT = 64, 48 +IMU_HZ = 200 +POSE_HZ = 30 +VIDEO_FPS = 30 + + +@lru_cache(maxsize=None) +def _h264_packets(n_frames: int) -> tuple[bytes, ...]: + """Annex B H.264, one frame per packet, keyframe every 2 frames, SPS/PPS repeated.""" + import av + + enc = av.CodecContext.create("libx264", "w") + enc.width, enc.height, enc.pix_fmt = WIDTH, HEIGHT, "yuv420p" + enc.time_base = Fraction(1, VIDEO_FPS) + enc.options = { + "bframes": "0", + "g": "2", + "repeat-headers": "1", + "tune": "zerolatency", + } + packets = [] + for i in range(n_frames): + img = np.full((HEIGHT, WIDTH, 3), i * 4 % 255, dtype=np.uint8) + frame = av.VideoFrame.from_ndarray(img, format="rgb24").reformat( + format="yuv420p" + ) + frame.pts = i + packets += [bytes(p) for p in enc.encode(frame)] + packets += [bytes(p) for p in enc.encode(None)] + return tuple(packets) + + +def _times(hz: float) -> list[int]: + return [T0 + round(i * 1e9 / hz) for i in range(int(DURATION_S * hz) + 1)] + + +def _sides(embodiment: str) -> tuple[str, ...]: + return spec.arm_sides(embodiment) + + +def _metadata(embodiment: str, declared: str) -> dict[str, str]: + info = { + "lab": "aria", + "site": "rl2", + "operator-id": "ab" * 32, + "task": "fold_clothes", + "instruction": "fold the shirt", + "embodiment": declared, + "rig-name": "test_rig", + "device-serial": "SN0001", + "scene": "kitchen_A", + "objects": "shirt,basket", + "start-time-unix": str(T0 // 10**6), + "end-time-unix": str((T0 + DURATION_S * 10**9) // 10**6), + "vendor-recording-id": "acme-rec-0001", + "vendor": "acme", + "spec-version": spec.SPEC_VERSION, + "license": "CC-BY-4.0", + } + if spec.profile_for(embodiment) is spec.HUMAN: + info["camera-timestamp-ref"] = "mid" + info["hand-joints"] = ",".join(spec.MANO_JOINTS) + info["shutter"] = "rolling:16ms" + return info + + +def _tf(links: list[tuple[str, str]], bad_rotation: bool) -> FrameTransforms: + tf = FrameTransforms() + for i, (parent, child) in enumerate(links): + x = tf.transforms.add(parent_frame_id=parent, child_frame_id=child) + x.translation.z = 0.1 + x.rotation.w = 2.0 if bad_rotation and i == 0 else 1.0 + return tf + + +def _rigid(z: float) -> list[float]: + T = np.eye(4) + T[:3, 3] = (0.3, 0.0, z) + return T.ravel().tolist() + + +def write_mcap_episode( + root: Path, + embodiment: str = "human_bimanual", + episode_id: str = "0", + *, + # 1. everything there + drop_topic: str | None = None, + drop_metadata_key: str | None = None, + declared_embodiment: str | None = None, + bad_site_slug: bool = False, + metadata_override: dict[str, str] | None = None, + unknown_hand_joint: bool = False, + empty_arm_pose: bool = False, + # 2. video + video_format: str = "h264", + skip_first_keyframe: bool = False, + calibration_width: int = WIDTH, + video_fps: int = VIDEO_FPS, + # 3. clocks + backwards_head_pose: bool = False, + hand_gap_s: float = 0.0, + device_clock_imu_info: bool = False, + # 4. calibration + unlink_main_camera: bool = False, + bad_tf_rotation: bool = False, + bad_upright_rotation: bool = False, + # 5. imu + imu_hz: int = IMU_HZ, + gravity_removed: bool = False, + # 6. poses + nan_head_pose: bool = False, + zeroed_left_hand: bool = False, + gripper_out_of_range: bool = False, + bad_arm_pose: bool = False, + bad_hand_confidence: bool = False, + hands_wrist_only: bool = False, + bad_wrist_pose: bool = False, + nan_eye_gaze: bool = False, + joint_names_mismatch: bool = False, + multi_frame_video: bool = False, + # 7. tasks & labels + labels_in: str | None = "annotation", + task_past_end: bool = False, + label_outside_tasks: bool = False, + stray_annotation_topic: bool = False, + label_crosses_tasks: bool = False, + labels_without_tasks: bool = False, + label_past_video: bool = False, + label_at_last_frame: bool = False, + annotation_metadata: bool = False, + calibration_twice: bool = False, + principal_x: float = WIDTH / 2, + repeat_frame_time: bool = False, + tamper: Callable | None = None, + distortion_model: str = "", + distortion_coeffs: tuple[float, ...] = (), + bad_annotation_type: bool = False, +) -> Path: + """Write `root/episode_/episode.mcap` and return the episode dir. + + With no switches set, the episode passes the validator. Each switch breaks one + thing: + + drop_topic leave this topic out (e.g. "/imu-info", "/main-camera-info") + drop_metadata_key leave this episode-info key out + declared_embodiment write this as the "embodiment" key, while the topics + follow `embodiment` + bad_site_slug "site" is "RL2 Lab" (not a lowercase slug) + metadata_override episode-info values to set as given + unknown_hand_joint "hand-joints" names "thumb_base", not in the vocabulary + empty_arm_pose robot: one arm's state `pose` left empty (as in ABC's + release) + video_format CompressedVideo.format on every camera + skip_first_keyframe cameras start on a P-frame + calibration_width width in /main-camera-info (frames stay 64 wide) + video_fps frame rate of every camera + backwards_head_pose two /head-pose messages swap timestamps + hand_gap_s hole of this length in /left-hand-keypoints + device_clock_imu_info /imu-info stamped with device time (1 s), not Unix time + unlink_main_camera /tf_static has no transform for the main camera + bad_tf_rotation first /tf_static transform has a non-unit quaternion + bad_upright_rotation every camera's -info-extra says to turn it 45 degrees + imu_hz /imu sample rate + gravity_removed /imu reads ~0 m/s^2 at rest + nan_head_pose one /head-pose position is NaN + zeroed_left_hand /left-hand-keypoints all zero with valid=true + gripper_out_of_range robot: one /-ee-state value is 1.5 + bad_arm_pose robot: one /-arm-state pose has a scaled rotation + bad_hand_confidence one /left-hand-keypoints confidence is 1.5 + hands_wrist_only valid: hands carry only wrist_pose, no keypoints + bad_wrist_pose one /left-hand-keypoints wrist_pose has a non-unit rotation + joint_names_mismatch robot: one arm's state lists 5 joint names for 6 joints + multi_frame_video /main-camera (or /top-camera) packs 2 frames per message + nan_eye_gaze one valid /eye-gaze message has a NaN yaw + labels_in where the label topics go: "annotation" (annotation.mcap), + "episode" (inline, as ABC does) or None (left out) + task_past_end the second /task ends 3 s after the recorded data + label_outside_tasks only the first /task is written, so the second label is + in no task + stray_annotation_topic annotation.mcap also carries /instruction + label_crosses_tasks the first label runs on into the second task + labels_without_tasks valid: labels but no /task (the episode info is the task) + label_past_video the point label sits after the last video frame + label_at_last_frame valid: the point label sits on the last video frame + annotation_metadata annotation.mcap carries its own session-metadata record + calibration_twice every camera's -info is published twice + principal_x cx in every camera's K (default: the image centre) + repeat_frame_time frame 21 of every camera repeats frame 20's timestamp + tamper tamper(topic, msg, t) -> None | (msg, t), applied to every + message before it is written (both files): any breakage + distortion_model, distortion_coeffs every camera's -info lens model and D + bad_annotation_type the labels are typed "summary" (only "language" is known) + """ + human = spec.profile_for(embodiment) is spec.HUMAN + sides = _sides(embodiment) + out = Path(root) / f"episode_{episode_id}" + out.mkdir(parents=True, exist_ok=True) + + info = _metadata(embodiment, declared_embodiment or embodiment) + if drop_metadata_key: + info.pop(drop_metadata_key) + if bad_site_slug: + info["site"] = "RL2 Lab" + info.update(metadata_override or {}) + if unknown_hand_joint: + info["hand-joints"] = info["hand-joints"].replace("thumb_cmc", "thumb_base") + + def keep(topic: str) -> bool: + return topic != drop_topic + + n_frames = DURATION_S * video_fps + packets = _h264_packets(n_frames) + if skip_first_keyframe: + packets = packets[1:] + video_times = [ + T0 + round((i + skip_first_keyframe) * 1e9 / video_fps) + for i in range(len(packets)) + ] + if repeat_frame_time: + video_times[21] = video_times[20] + + if human: + cameras = {"/main-camera": "main-camera"} + links = [("head", "main-camera"), ("head", "imu"), ("head", "et")] + if unlink_main_camera: + links.remove(("head", "main-camera")) + else: + cameras = {"/top-camera": "top-camera"} + links = [("world", "top-camera")] + for s in sides: + cameras[f"/{s}-wrist-camera"] = f"{s}-wrist-camera" + links += [("world", f"{s}_arm_base"), (f"{s}_ee", f"{s}-wrist-camera")] + + with open(out / spec.EPISODE_FILE, "wb") as f, Writer(f) as w: + + def write(topic: str, msg, t: int) -> None: + if tamper and (changed := tamper(topic, msg, t)): + msg, t = changed + if keep(topic): + w.write_message(topic, msg, log_time=t, publish_time=t) + + # mcap_protobuf's Writer has no metadata call; its underlying mcap writer does. + w._writer.add_metadata(spec.METADATA_RECORD, info) + + for topic, frame_id in cameras.items(): + width = calibration_width if topic == "/main-camera" else WIDTH + write( + topic + spec.CALIBRATION_SUFFIX, + CameraCalibration( + frame_id=frame_id, + width=width, + height=HEIGHT, + K=[50, 0, principal_x, 0, 50, HEIGHT / 2, 0, 0, 1], + distortion_model=distortion_model, + D=list(distortion_coeffs), + ), + T0, + ) + if calibration_twice: + write( + topic + spec.CALIBRATION_SUFFIX, + CameraCalibration(frame_id=frame_id, width=width, height=HEIGHT), + T0 + 1, + ) + write( + topic + spec.CAMERA_EXTRA_SUFFIX, + schemas.CameraInfoExtra( + frame_id=frame_id, + valid_radius=0.0, + upright_rotation_cw_deg=45 if bad_upright_rotation else 0, + ), + T0, + ) + pairs = list(zip(packets, video_times)) + if multi_frame_video and topic in ("/main-camera", "/top-camera"): + pairs = [ + (b"".join(d for d, _ in pairs[i : i + 2]), pairs[i][1]) + for i in range(0, len(pairs), 2) + ] + for data, t in pairs: + write( + topic, + CompressedVideo(frame_id=frame_id, data=data, format=video_format), + t, + ) + + write("/tf_static", _tf(links, bad_tf_rotation), T0) + write("/instruction", schemas.Instruction(data=info["instruction"]), T0) + + if human: + _write_human( + write, + sides, + backwards_head_pose=backwards_head_pose, + hand_gap_s=hand_gap_s, + device_clock_imu_info=device_clock_imu_info, + imu_hz=imu_hz, + gravity_removed=gravity_removed, + nan_head_pose=nan_head_pose, + zeroed_left_hand=zeroed_left_hand, + bad_hand_confidence=bad_hand_confidence, + hands_wrist_only=hands_wrist_only, + bad_wrist_pose=bad_wrist_pose, + nan_eye_gaze=nan_eye_gaze, + ) + else: + _write_robot( + write, + sides, + empty_arm_pose=empty_arm_pose, + gripper_out_of_range=gripper_out_of_range, + bad_arm_pose=bad_arm_pose, + joint_names_mismatch=joint_names_mismatch, + ) + if labels_in == "episode": + _write_labels( + write, + task_past_end, + label_outside_tasks, + label_crosses_tasks, + labels_without_tasks, + label_past_video, + label_at_last_frame, + bad_annotation_type, + ) + + if labels_in == "annotation": + with open(out / spec.ANNOTATION_FILE, "wb") as f, Writer(f) as w: + + def write_label(topic: str, msg, t: int) -> None: + if tamper and (changed := tamper(topic, msg, t)): + msg, t = changed + w.write_message(topic, msg, log_time=t, publish_time=t) + + _write_labels( + write_label, + task_past_end, + label_outside_tasks, + label_crosses_tasks, + labels_without_tasks, + label_past_video, + label_at_last_frame, + bad_annotation_type, + ) + if annotation_metadata: + w._writer.add_metadata(spec.METADATA_RECORD, {"lab": "other"}) + if stray_annotation_topic: + write_label("/instruction", schemas.Instruction(data="fold"), T0) + return out + + +def _write_labels( + write, + task_past_end: bool, + label_outside_tasks: bool, + label_crosses_tasks: bool, + labels_without_tasks: bool, + label_past_video: bool, + label_at_last_frame: bool, + bad_annotation_type: bool, +) -> None: + """Two back-to-back tasks, one ranged label inside the first and a point label + inside the second.""" + half = DURATION_S * 10**9 // 2 + end = T0 + 2 * half + (3 * 10**9 if task_past_end else 0) + tasks = [ + (T0, T0 + half, "fold_clothes", ["shirt"]), + (T0 + half, end, "pack_clothes", ["shirt", "basket"]), + ] + for start, stop, name, objects in tasks[ + : 0 if labels_without_tasks else 1 if label_outside_tasks else 2 + ]: + write( + "/task", + schemas.Task( + name=name, + description=name.replace("_", " "), + objects=objects, + end_time_unix_ns=stop, + ), + start, + ) + kind = "summary" if bad_annotation_type else "language" + first_end = T0 + half + half // 2 if label_crosses_tasks else T0 + half // 2 + write( + "/subtask-annotation", + schemas.Annotation( + data="pick up the shirt", + end_time_unix_ns=first_end, + annotation_type=kind, + annotation_vendor="acme", + ), + T0 + half // 5, + ) + write( + "/subtask-annotation", + schemas.Annotation(data="drop it in the basket", annotation_type=kind), + T0 + 2 * half - 10**6 + if label_past_video + else T0 + round((DURATION_S * VIDEO_FPS - 1) * 1e9 / VIDEO_FPS) + if label_at_last_frame + else T0 + half + half // 3, + ) + + +def _write_human( + write, + sides, + *, + backwards_head_pose, + hand_gap_s, + device_clock_imu_info, + imu_hz, + gravity_removed, + nan_head_pose, + zeroed_left_hand, + bad_hand_confidence, + hands_wrist_only, + bad_wrist_pose, + nan_eye_gaze, +) -> None: + write( + "/imu-info", + schemas.ImuInfo( + frame_id="imu", + accel_noise_density=1e-3, + accel_random_walk=1e-4, + gyro_noise_density=1e-4, + gyro_random_walk=1e-5, + ), + 10**9 if device_clock_imu_info else T0, + ) + for i, t in enumerate(_times(imu_hz)): + m = schemas.Imu(frame_id="imu") + m.linear_acceleration.z = 0.02 if gravity_removed else spec.GRAVITY + m.angular_velocity.x = 0.01 * np.sin(i / 20) + write("/imu", m, t) + + pose_times = _times(POSE_HZ) + if backwards_head_pose: + pose_times[10], pose_times[11] = pose_times[11], pose_times[10] + for i, t in enumerate(pose_times): + p = PoseInFrame(frame_id=spec.WORLD_FRAME) + p.pose.position.z = 1.5 + p.pose.orientation.w = 1.0 + if nan_head_pose and i == 5: + p.pose.position.x = float("nan") + write("/head-pose", p, t) + + for i, t in enumerate(_times(POSE_HZ)): + g = schemas.EyeGaze(frame_id="et", yaw=0.1, pitch=-0.3, depth=0.8, valid=True) + if nan_eye_gaze and i == 5: + g.yaw = float("nan") + write("/eye-gaze", g, t) + + ok = schemas.TrackingQuality.OK + hand = np.linspace(0.1, 0.5, 63).tolist() + for t in _times(POSE_HZ): + write("/head-pose-quality", schemas.TrackingQuality(state=ok), t) + for s in sides: + in_gap = s == "left" and T0 + 0.5e9 < t < T0 + (0.5 + hand_gap_s) * 1e9 + if in_gap: + continue + zero = zeroed_left_hand and s == "left" + k = schemas.HandKeypoints( + frame_id=spec.WORLD_FRAME, + positions=[] if hands_wrist_only else [0.0] * 63 if zero else hand, + confidence=1.5 + if bad_hand_confidence and s == "left" and t == T0 + else 0.9, + joint_confidence=[] if hands_wrist_only else [0.9] * 21, + valid=True, + ) + k.wrist_pose.position.z = 1.2 + k.wrist_pose.orientation.w = ( + 2.0 if bad_wrist_pose and s == "left" and t == T0 else 1.0 + ) + write(f"/{s}-hand-keypoints", k, t) + + +def _write_robot( + write, + sides, + *, + empty_arm_pose, + gripper_out_of_range, + bad_arm_pose, + joint_names_mismatch, +): + joints = [0.1, -0.2, 0.3, 0.0, 0.5, -0.1] + names = [f"joint{j}" for j in range(1, 7)] + for i, t in enumerate(_times(POSE_HZ)): + for s in sides: + pose = _rigid(0.2 + 0.001 * i) + if bad_arm_pose and s == sides[-1] and i == 5: + pose[0] = 2.0 # R[0, 0]: no longer a rotation + last = s == sides[-1] + write( + f"/{s}-arm-state", + schemas.ArmState( + frame_id=f"{s}_arm_base", + joint_names=names[:5] if joint_names_mismatch and last else names, + position=joints, + velocity=[0.0] * 6, + torque=[0.5] * 6, + ee_pose=[] if empty_arm_pose and last else pose, + ), + t, + ) + write( + f"/{s}-arm-action", + schemas.ArmAction( + frame_id=f"{s}_arm_base", + joint_names=names, + position=joints, + ee_pose=_rigid(0.2 + 0.001 * (i + 1)), + ), + t, + ) + g = 1.5 if gripper_out_of_range and last and i == 5 else 0.4 + write(f"/{s}-ee-state", schemas.GripperState(position=g), t) + write(f"/{s}-ee-action", schemas.GripperState(position=0.4), t) diff --git a/tests/integration/test_mcap_abc.py b/tests/integration/test_mcap_abc.py new file mode 100644 index 0000000000..2fa1766e44 --- /dev/null +++ b/tests/integration/test_mcap_abc.py @@ -0,0 +1,46 @@ +"""Vendor MCAP validator on real episodes. + + EGOVERSE_TEST_ABC_DIR= pytest --integration tests/integration/test_mcap_abc.py + EGOVERSE_TEST_MCAP_DIR= pytest --integration tests/integration/test_mcap_abc.py + +ABC-130k predates the spec (own robot types, no /tf_static or full episode info), so its +episodes must be read end to end and reported, not pass. Episodes delivered to the spec +(e.g. Aria recordings converted to it) must pass. +""" + +import os +from pathlib import Path + +import pytest + +from egomimic.rldb.mcap import spec +from egomimic.rldb.mcap.validate import validate_episode + + +def _episodes(var: str) -> list[Path]: + root = os.environ.get(var) + if not root: + pytest.skip(f"set {var} to a directory of episode_*/ dirs") + eps = sorted(p for p in Path(root).glob("episode_*") if p.is_dir()) + if not eps: + pytest.skip(f"no episode_*/ dirs under {root}") + return eps + + +def test_abc_episodes_are_read_and_reported(): + for ep in _episodes("EGOVERSE_TEST_ABC_DIR"): + r = validate_episode(ep) + unreadable = [p for p in r.problems if p.message.startswith("cannot read")] + assert not unreadable, f"{ep}: {unreadable}" + assert r.problems, f"{ep}: ABC predates the spec, expected problems" + print(f"{ep.name}: {len(r.problems)} problems") + for p in r.problems: + print(f" {p}") + + +def test_spec_deliveries_pass(): + for ep in _episodes("EGOVERSE_TEST_MCAP_DIR"): + r = validate_episode(ep) + assert r.problems == [], f"{ep}:\n" + "\n".join(map(str, r.problems)) + assert r.recording_id, ep + assert (ep / spec.EPISODE_FILE).exists() diff --git a/tests/unit/test_mcap_validate.py b/tests/unit/test_mcap_validate.py new file mode 100644 index 0000000000..cca8487d6a --- /dev/null +++ b/tests/unit/test_mcap_validate.py @@ -0,0 +1,284 @@ +"""Vendor MCAP validator (egomimic/rldb/mcap) on tiny synthetic episodes. + +Valid episodes of every profile pass; each breakage in fixtures.mcap_episodes produces +exactly its expected problems and nothing else. +""" + +from pathlib import Path + +import pytest +from fixtures.mcap_episodes import T0, write_mcap_episode + +from egomimic.rldb.mcap import schemas, spec +from egomimic.rldb.mcap.validate import main, validate_episode + +PROTO = Path(schemas.__file__).parent / "proto" / "egoverse.proto" + + +def problems(tmp_path, embodiment="human_bimanual", **kw) -> list[tuple]: + r = validate_episode(write_mcap_episode(tmp_path, embodiment, **kw)) + return [(p.group, p.topic, p.message) for p in r.problems] + + +@pytest.mark.parametrize( + "embodiment, kw", + [ + ("human_bimanual", {}), + ("human_right_arm", {}), + ("human_left_arm", {}), + ("eva_bimanual", {}), + ("eva_left_arm", {}), + ("human_bimanual", {"labels_in": "episode"}), + ("eva_bimanual", {"labels_in": None}), + ("human_bimanual", {"hands_wrist_only": True}), + ("human_bimanual", {"drop_topic": "/main-camera-info-extra"}), + ("human_bimanual", {"labels_without_tasks": True}), + ("human_bimanual", {"labels_without_tasks": True, "label_at_last_frame": True}), + ( + "human_bimanual", + {"distortion_model": "fisheye624", "distortion_coeffs": (0.1,) * 12}, + ), + ], +) +def test_valid_episode_passes(tmp_path, embodiment, kw): + r = validate_episode(write_mcap_episode(tmp_path, embodiment, **kw)) + assert r.problems == [] + assert r.embodiment == embodiment + assert r.recording_id == spec.recording_id("aria", "acme-rec-0001") + + +# (embodiment, breakage, [(group, topic, substring of the message), ...]) +BREAKAGES = [ + # 1. everything there + ("human_bimanual", {"drop_topic": "/imu-info"}, [("present", "/imu-info", "missing required topic")]), + ("human_bimanual", {"drop_metadata_key": "vendor"}, [("present", None, "'vendor' missing")]), + ("human_bimanual", {"drop_metadata_key": "site"}, [("present", None, "'site' missing")]), + ("human_bimanual", {"drop_metadata_key": "license"}, [("present", None, "'license' missing")]), + ("human_bimanual", {"drop_metadata_key": "vendor-recording-id"}, [("present", None, "'vendor-recording-id' missing")]), + ("human_bimanual", {"drop_metadata_key": "hand-joints"}, [("present", None, "'hand-joints' missing")]), + ("human_bimanual", {"bad_site_slug": True}, [("present", None, "not a lowercase slug")]), + ("human_bimanual", {"metadata_override": {"operator-id": "jane.doe@gatech.edu"}}, [("present", None, "not a SHA-256 hex digest")]), + ("human_bimanual", {"metadata_override": {"shutter": "rolling"}}, [("present", None, "expected 'global' or 'rolling:ms'")]), + ("human_bimanual", {"metadata_override": {"camera-timestamp-ref": "middle"}}, [("present", None, "'camera-timestamp-ref' = 'middle'")]), + ("human_bimanual", {"annotation_metadata": True}, [("present", None, "annotation.mcap carries metadata records")]), + ("human_bimanual", {"metadata_override": {"start-time-unix": "--1760000000000"}}, [("present", None, "is not an integer")]), + ("human_bimanual", {"metadata_override": {"end-time-unix": "1760000002000²"}}, [("present", None, "is not an integer")]), + ("human_bimanual", {"unknown_hand_joint": True}, [("present", None, "unknown joints ['thumb_base']")]), + ("human_bimanual", {"declared_embodiment": "robot_dog"}, [("present", None, "'embodiment' = 'robot_dog'")]), + ("human_bimanual", {"declared_embodiment": "human_right_arm"}, [("present", "/left-hand-keypoints", "does not fit embodiment")]), + ("human_bimanual", {"stray_annotation_topic": True}, [("present", "/instruction", "belong in annotation.mcap")]), + ("eva_bimanual", {"empty_arm_pose": True}, [ + ("present", "/right-arm-state", "'ee_pose' empty on 61/61"), + ("poses", "/right-arm-state", "not a finite rigid 4x4"), + ]), + # 2. video + ("human_bimanual", {"video_format": "vp9"}, [("video", "/main-camera", "format 'vp9'")]), + ("human_bimanual", {"skip_first_keyframe": True}, [("video", "/main-camera", "not a keyframe")]), + ("human_bimanual", {"calibration_width": 128}, [("video", "/main-camera", "calibration says 128x48")]), + ("human_bimanual", {"video_fps": 15}, [("video", "/main-camera", "15.0 fps")]), + ("human_bimanual", {"multi_frame_video": True}, [("video", "/main-camera", "exactly one frame")]), + ("eva_bimanual", {"multi_frame_video": True}, [("video", "/top-camera", "exactly one frame")]), + # 3. clocks + ("human_bimanual", {"backwards_head_pose": True}, [("clocks", "/head-pose", "backwards 1 times")]), + ("human_bimanual", {"hand_gap_s": 0.8}, [("clocks", "/left-hand-keypoints", "0.80 s gap")]), + ("human_bimanual", {"device_clock_imu_info": True}, [("clocks", "/imu-info", "before 2015")]), + ("human_bimanual", {"repeat_frame_time": True}, [("clocks", "/main-camera", "1 frames repeat a timestamp")]), + # 4. calibration + ("human_bimanual", {"drop_topic": "/main-camera-info"}, [("calibration", "/main-camera", "no /main-camera-info")]), + ("human_bimanual", {"unlink_main_camera": True}, [("calibration", "/main-camera", "no path to 'world'")]), + ("human_bimanual", {"bad_tf_rotation": True}, [("calibration", "/tf_static", "non-unit rotation")]), + ("human_bimanual", {"bad_upright_rotation": True}, [("calibration", "/main-camera", "upright_rotation_cw_deg 45")]), + ("human_bimanual", {"calibration_twice": True}, [("calibration", "/main-camera-info", "got 2 foxglove.CameraCalibration")]), + ("human_bimanual", {"principal_x": 500.0}, [("calibration", "/main-camera", "principal point (500, 24) is outside")]), + ("human_bimanual", {"distortion_model": "equidistant", "distortion_coeffs": (0.1,) * 4}, [("calibration", "/main-camera", "distortion_model 'equidistant'")]), + ("human_bimanual", {"distortion_model": "fisheye624", "distortion_coeffs": (0.1,) * 5}, [("calibration", "/main-camera", "needs 12 D coefficients, got 5")]), + # 5. imu + ("human_bimanual", {"imu_hz": 100}, [("imu", "/imu", "100 Hz")]), + ("human_bimanual", {"gravity_removed": True}, [("imu", "/imu", "gravity removed")]), + # 6. poses + ("human_bimanual", {"nan_head_pose": True}, [("poses", "/head-pose", "1/61 poses")]), + ("human_bimanual", {"zeroed_left_hand": True}, [("poses", "/left-hand-keypoints", "all-zero with valid=true")]), + ("human_bimanual", {"bad_hand_confidence": True}, [("poses", "/left-hand-keypoints", "confidence outside [0, 1]")]), + ("human_bimanual", {"bad_wrist_pose": True}, [("poses", "/left-hand-keypoints", "wrist poses")]), + ("human_bimanual", {"nan_eye_gaze": True}, [("poses", "/eye-gaze", "NaN/Inf")]), + ("eva_bimanual", {"gripper_out_of_range": True}, [("poses", "/right-ee-state", "outside [0, 1]")]), + ("eva_bimanual", {"bad_arm_pose": True}, [("poses", "/right-arm-state", "1/61 poses")]), + ("eva_bimanual", {"joint_names_mismatch": True}, [("poses", "/right-arm-state", "do not match joint_names")]), + # 7. tasks & labels + ("human_bimanual", {"task_past_end": True}, [("labels", "/task", "outside the recorded data")]), + ("eva_bimanual", {"label_outside_tasks": True}, [("labels", "/subtask-annotation", "inside one /task")]), + ("human_bimanual", {"label_crosses_tasks": True}, [("labels", "/subtask-annotation", "inside one /task")]), + ("human_bimanual", {"bad_annotation_type": True}, [("labels", "/subtask-annotation", "['summary']")]), + ("human_bimanual", {"labels_without_tasks": True, "label_past_video": True}, [("labels", "/subtask-annotation", "outside /main-camera's frames")]), +] # fmt: skip + + +@pytest.mark.parametrize( + "embodiment, kw, expected", + BREAKAGES, + ids=[f"{e}-{next(iter(kw))}={next(iter(kw.values()))}" for e, kw, _ in BREAKAGES], +) +def test_breakage_reports_exactly_its_problem(tmp_path, embodiment, kw, expected): + found = problems(tmp_path, embodiment, **kw) + assert len(found) == len(expected), found + for (group, topic, message), (g, t, sub) in zip(sorted(found), sorted(expected)): + assert (group, topic) == (g, t), found + assert sub in message, found + + +def test_cli_pass_fail_and_summary(tmp_path, capsys): + good = write_mcap_episode(tmp_path, episode_id="good") + bad = write_mcap_episode(tmp_path, episode_id="bad", imu_hz=100) + assert main([str(good)]) == 0 + assert main([str(good), str(bad)]) == 1 + out = capsys.readouterr().out + assert f"PASS {good}" in out + assert f"FAIL {bad}" in out and "[imu] /imu: 100 Hz" in out + assert "1/2 passed" in out + assert spec.recording_id("aria", "acme-rec-0001") in out + + +def test_missing_and_unreadable_episodes(tmp_path): + r = validate_episode(tmp_path / "episode_nope") + assert [p.message for p in r.problems] == [ + f"no episode.mcap in {tmp_path / 'episode_nope'}" + ] + junk = tmp_path / "episode_junk" + junk.mkdir() + (junk / "episode.mcap").write_bytes(b"not an mcap") + (p,) = validate_episode(junk).problems + assert p.message.startswith("cannot read episode.mcap") + + +def test_unreadable_annotation_file(tmp_path): + ep = write_mcap_episode(tmp_path, labels_in=None) + (ep / spec.ANNOTATION_FILE).write_bytes(b"garbage") + (p,) = validate_episode(ep).problems + assert p.message.startswith("cannot read annotation.mcap") + + +def test_proto_file_matches_schemas(): + assert PROTO.read_text() == schemas.render_proto(), ( + "regenerate: python -m egomimic.rldb.mcap.schemas > " + str(PROTO) + ) + + +def test_embodiments_match_enum(): + from egomimic.rldb.embodiment.embodiment import EMBODIMENT + + assert set(spec.EMBODIMENTS) == {e.name.lower() for e in EMBODIMENT} + + +def test_mano_joints_match_human_finger_edges(): + # Canonical MANO: wrist, then 4 joints per finger, thumb to pinky. + assert len(spec.MANO_JOINTS) == 21 and spec.MANO_JOINTS[0] == "wrist" + assert spec.MANO_JOINTS[1:5] == ("thumb_cmc", "thumb_mcp", "thumb_ip", "thumb_tip") + assert spec.MANO_JOINTS[17:] == ("pinky_mcp", "pinky_pip", "pinky_dip", "pinky_tip") + + +def test_lance_ids_and_annotation_types_match_lance_schema(): + lance_schema = pytest.importorskip("egomimic.rldb.lance.schema") + assert spec.recording_id("aria", "x") == lance_schema.recording_id("aria", "x") + if hasattr(lance_schema, "ANNOTATION_TYPES"): + assert spec.ANNOTATION_TYPES == lance_schema.ANNOTATION_TYPES + + +def _late_hand_frame(topic, msg, t): + if topic == "/left-hand-keypoints" and t > T0 + 10**9: + msg.frame_id = "tracker" # message 0 is fine; later ones are not linked + return msg, t + + +def _head_in_odom(topic, msg, t): + if topic == "/head-pose": + msg.frame_id = "odom" + return msg, t + if ( + topic == "/tf_static" + ): # odom does reach world, but /head-pose must be T_world_head + x = msg.transforms.add(parent_frame_id="world", child_frame_id="odom") + x.rotation.w = 1.0 + return msg, t + + +def _nan_static_translation(topic, msg, t): + if topic == "/tf_static": + msg.transforms[0].translation.x = float("nan") + return msg, t + + +def _main_camera_fixed_in_world(topic, msg, t): + """A consistent camera frame, rigidly mounted to the world instead of the head.""" + if topic.startswith("/main-camera"): + msg.frame_id = "room_cam" + return msg, t + if topic == "/tf_static": + x = msg.transforms.add(parent_frame_id="world", child_frame_id="room_cam") + x.rotation.w = 1.0 + return msg, t + + +def _point_label_at_task_end(topic, msg, t): + if topic == "/subtask-annotation" and not msg.end_time_unix_ns: + return msg, T0 + 10**9 - 10_000_000 # 10 ms before task 1 ends: not a frame + + +@pytest.mark.parametrize( + "tamper, expected", + [ + ( + _late_hand_frame, + ("calibration", "/left-hand-keypoints", "'tracker' has no path"), + ), + (_head_in_odom, ("calibration", "/head-pose", "must be in frame 'world'")), + ( + _nan_static_translation, + ("calibration", "/tf_static", "1 transforms have a NaN / Inf translation"), + ), + ( + _main_camera_fixed_in_world, + ("calibration", "/main-camera", "no fixed path to 'head'"), + ), + ( + _point_label_at_task_end, + ("labels", "/subtask-annotation", "inside one /task"), + ), + ], +) +def test_tampered_episode_reports_exactly_its_problem(tmp_path, tamper, expected): + found = problems(tmp_path, tamper=tamper) + assert len(found) == 1, found + group, topic, message = found[0] + assert (group, topic) == expected[:2] and expected[2] in message, found + + +def test_calibration_naming_another_frame_than_the_video_is_caught(tmp_path): + """The calibration (and -info-extra) moved to `world`, the video frames not: the + camera has two frames, and the steps camera is no longer on the head.""" + + def calib_in_world(topic, msg, t): + if topic in ("/main-camera-info", "/main-camera-info-extra"): + msg.frame_id = "world" + return msg, t + + found = problems(tmp_path, tamper=calib_in_world) + assert ("calibration", "/main-camera") == found[0][:2] + assert any("video frames name ['main_cam']" in m or "video frames name" in m + for *_, m in found), found # fmt: skip + assert any("no fixed path to 'head'" in m for *_, m in found), found + + +def test_wrong_message_type_is_reported_not_raised(tmp_path): + from egomimic.rldb.mcap import schemas + + def imu_as_instruction(topic, msg, t): + if topic == "/imu": + return schemas.Instruction(data="not an imu"), t + + found = problems(tmp_path, tamper=imu_as_instruction) + assert ( + "present", + "/imu", + "type is 'egoverse.Instruction', expected 'egoverse.Imu'", + ) in found + assert any(g == "imu" and "could not check" in m for g, _, m in found), found diff --git a/tests/unit/test_packaging.py b/tests/unit/test_packaging.py index dd035fc212..bcca6ae48a 100644 --- a/tests/unit/test_packaging.py +++ b/tests/unit/test_packaging.py @@ -77,3 +77,8 @@ def test_wheel_ships_robot_resources(wheel_files): # XML relative to egomimic.__file__. assert "egomimic/resources/model_x5.xml" in wheel_files assert "egomimic/resources/model_arx.urdf" in wheel_files + + +def test_wheel_ships_mcap_proto(wheel_files): + # The .proto handed to vendors; egomimic.rldb.mcap.schemas renders the same text. + assert "egomimic/rldb/mcap/proto/egoverse.proto" in wheel_files diff --git a/uv.lock b/uv.lock index 13130da7b9..81faeb6b80 100644 --- a/uv.lock +++ b/uv.lock @@ -6,10 +6,10 @@ resolution-markers = [ "python_full_version >= '3.14' and sys_platform == 'emscripten'", "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'", "python_full_version == '3.13.*' and sys_platform == 'win32'", - "python_full_version == '3.12.*' and sys_platform == 'win32'", "python_full_version == '3.13.*' and sys_platform == 'emscripten'", - "python_full_version == '3.12.*' and sys_platform == 'emscripten'", "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.12.*' and sys_platform == 'win32'", + "python_full_version == '3.12.*' and sys_platform == 'emscripten'", "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", "python_full_version < '3.12' and sys_platform == 'win32'", "python_full_version < '3.12' and sys_platform == 'emscripten'", @@ -1030,6 +1030,7 @@ dependencies = [ { name = "datasets" }, { name = "dm-control" }, { name = "einops" }, + { name = "foxglove-schemas-protobuf" }, { name = "geomloss" }, { name = "gpustat" }, { name = "h5py" }, @@ -1038,6 +1039,8 @@ dependencies = [ { name = "ipython" }, { name = "lightning" }, { name = "matplotlib" }, + { name = "mcap" }, + { name = "mcap-protobuf-support" }, { name = "mediapy" }, { name = "mink" }, { name = "mujoco" }, @@ -1092,6 +1095,7 @@ requires-dist = [ { name = "datasets", specifier = "==4.0.0" }, { name = "dm-control", specifier = "==1.0.8" }, { name = "einops" }, + { name = "foxglove-schemas-protobuf", specifier = "==0.4.0" }, { name = "geomloss" }, { name = "gpustat" }, { name = "h5py" }, @@ -1100,6 +1104,8 @@ requires-dist = [ { name = "ipython" }, { name = "lightning" }, { name = "matplotlib" }, + { name = "mcap", specifier = "==1.5.0" }, + { name = "mcap-protobuf-support", specifier = "==0.5.4" }, { name = "mediapy" }, { name = "mink", specifier = "==0.0.13" }, { name = "mujoco", specifier = "==3.4.0" }, @@ -1243,6 +1249,18 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/fd/ba/56147c165442cc5ba7e82ecf301c9a68353cede498185869e6e02b4c264f/fonttools-4.62.1-py3-none-any.whl", hash = "sha256:7487782e2113861f4ddcc07c3436450659e3caa5e470b27dc2177cade2d8e7fd", size = 1152647, upload-time = "2026-03-13T13:54:22.735Z" }, ] +[[package]] +name = "foxglove-schemas-protobuf" +version = "0.4.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "protobuf" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/47/d8/7ca265b4721e2ae912cd0fa5cc96d90357b5ae1ae15c5dff653bf9f22ec5/foxglove_schemas_protobuf-0.4.0.tar.gz", hash = "sha256:3fc9610d7ec6cc4b4029f746ab9699a4ad396bdd7551b99d2a37509d13ef3cc2", size = 31026, upload-time = "2026-07-29T20:46:47.162Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b1/ff/67f49b46e40dca6fb5e5077c3f665c07155f5a878cf349185957d075e999/foxglove_schemas_protobuf-0.4.0-py3-none-any.whl", hash = "sha256:e6a35df574f4c10ea455e8660abd09af028ea00d90e01c78f8696972e2decc95", size = 98809, upload-time = "2026-07-29T20:46:45.863Z" }, +] + [[package]] name = "fqdn" version = "1.5.1" @@ -1864,7 +1882,7 @@ name = "jinxed" version = "2.0.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "ansicon", marker = "sys_platform == 'win32'" }, + { name = "ansicon" }, ] sdist = { url = "https://files.pythonhosted.org/packages/cb/eb/8821ce6e7386e96355f2c6be83944925b4a0870572896fd33a4e61b8aa5a/jinxed-2.0.0.tar.gz", hash = "sha256:64b960b8f8d9966e1b2e7cb57ea2b1aa0a8d7e68045b96dc7ef58201e43a1209", size = 127118, upload-time = "2026-05-08T21:25:25.917Z" } wheels = [ @@ -2549,6 +2567,54 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/40/44/3ee09a5b60cb44c4f2fbc1c9015cfd6ff5afc08f991cab295d3024dcbf2d/lxml-6.1.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:7da13bb6fbadfafb474e0226a30570a3445cfd47c86296f2446dafbd77079ace", size = 3508860, upload-time = "2026-04-18T04:32:48.619Z" }, ] +[[package]] +name = "lz4" +version = "4.4.5" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/57/51/f1b86d93029f418033dddf9b9f79c8d2641e7454080478ee2aab5123173e/lz4-4.4.5.tar.gz", hash = "sha256:5f0b9e53c1e82e88c10d7c180069363980136b9d7a8306c4dca4f760d60c39f0", size = 172886, upload-time = "2025-11-03T13:02:36.061Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/93/5b/6edcd23319d9e28b1bedf32768c3d1fd56eed8223960a2c47dacd2cec2af/lz4-4.4.5-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:d6da84a26b3aa5da13a62e4b89ab36a396e9327de8cd48b436a3467077f8ccd4", size = 207391, upload-time = "2025-11-03T13:01:36.644Z" }, + { url = "https://files.pythonhosted.org/packages/34/36/5f9b772e85b3d5769367a79973b8030afad0d6b724444083bad09becd66f/lz4-4.4.5-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:61d0ee03e6c616f4a8b69987d03d514e8896c8b1b7cc7598ad029e5c6aedfd43", size = 207146, upload-time = "2025-11-03T13:01:37.928Z" }, + { url = "https://files.pythonhosted.org/packages/04/f4/f66da5647c0d72592081a37c8775feacc3d14d2625bbdaabd6307c274565/lz4-4.4.5-cp311-cp311-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:33dd86cea8375d8e5dd001e41f321d0a4b1eb7985f39be1b6a4f466cd480b8a7", size = 1292623, upload-time = "2025-11-03T13:01:39.341Z" }, + { url = "https://files.pythonhosted.org/packages/85/fc/5df0f17467cdda0cad464a9197a447027879197761b55faad7ca29c29a04/lz4-4.4.5-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:609a69c68e7cfcfa9d894dc06be13f2e00761485b62df4e2472f1b66f7b405fb", size = 1279982, upload-time = "2025-11-03T13:01:40.816Z" }, + { url = "https://files.pythonhosted.org/packages/25/3b/b55cb577aa148ed4e383e9700c36f70b651cd434e1c07568f0a86c9d5fbb/lz4-4.4.5-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:75419bb1a559af00250b8f1360d508444e80ed4b26d9d40ec5b09fe7875cb989", size = 1368674, upload-time = "2025-11-03T13:01:42.118Z" }, + { url = "https://files.pythonhosted.org/packages/fb/31/e97e8c74c59ea479598e5c55cbe0b1334f03ee74ca97726e872944ed42df/lz4-4.4.5-cp311-cp311-win32.whl", hash = "sha256:12233624f1bc2cebc414f9efb3113a03e89acce3ab6f72035577bc61b270d24d", size = 88168, upload-time = "2025-11-03T13:01:43.282Z" }, + { url = "https://files.pythonhosted.org/packages/18/47/715865a6c7071f417bef9b57c8644f29cb7a55b77742bd5d93a609274e7e/lz4-4.4.5-cp311-cp311-win_amd64.whl", hash = "sha256:8a842ead8ca7c0ee2f396ca5d878c4c40439a527ebad2b996b0444f0074ed004", size = 99491, upload-time = "2025-11-03T13:01:44.167Z" }, + { url = "https://files.pythonhosted.org/packages/14/e7/ac120c2ca8caec5c945e6356ada2aa5cfabd83a01e3170f264a5c42c8231/lz4-4.4.5-cp311-cp311-win_arm64.whl", hash = "sha256:83bc23ef65b6ae44f3287c38cbf82c269e2e96a26e560aa551735883388dcc4b", size = 91271, upload-time = "2025-11-03T13:01:45.016Z" }, + { url = "https://files.pythonhosted.org/packages/1b/ac/016e4f6de37d806f7cc8f13add0a46c9a7cfc41a5ddc2bc831d7954cf1ce/lz4-4.4.5-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:df5aa4cead2044bab83e0ebae56e0944cc7fcc1505c7787e9e1057d6d549897e", size = 207163, upload-time = "2025-11-03T13:01:45.895Z" }, + { url = "https://files.pythonhosted.org/packages/8d/df/0fadac6e5bd31b6f34a1a8dbd4db6a7606e70715387c27368586455b7fc9/lz4-4.4.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:6d0bf51e7745484d2092b3a51ae6eb58c3bd3ce0300cf2b2c14f76c536d5697a", size = 207150, upload-time = "2025-11-03T13:01:47.205Z" }, + { url = "https://files.pythonhosted.org/packages/b7/17/34e36cc49bb16ca73fb57fbd4c5eaa61760c6b64bce91fcb4e0f4a97f852/lz4-4.4.5-cp312-cp312-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:7b62f94b523c251cf32aa4ab555f14d39bd1a9df385b72443fd76d7c7fb051f5", size = 1292045, upload-time = "2025-11-03T13:01:48.667Z" }, + { url = "https://files.pythonhosted.org/packages/90/1c/b1d8e3741e9fc89ed3b5f7ef5f22586c07ed6bb04e8343c2e98f0fa7ff04/lz4-4.4.5-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2c3ea562c3af274264444819ae9b14dbbf1ab070aff214a05e97db6896c7597e", size = 1279546, upload-time = "2025-11-03T13:01:50.159Z" }, + { url = "https://files.pythonhosted.org/packages/55/d9/e3867222474f6c1b76e89f3bd914595af69f55bf2c1866e984c548afdc15/lz4-4.4.5-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:24092635f47538b392c4eaeff14c7270d2c8e806bf4be2a6446a378591c5e69e", size = 1368249, upload-time = "2025-11-03T13:01:51.273Z" }, + { url = "https://files.pythonhosted.org/packages/b2/e7/d667d337367686311c38b580d1ca3d5a23a6617e129f26becd4f5dc458df/lz4-4.4.5-cp312-cp312-win32.whl", hash = "sha256:214e37cfe270948ea7eb777229e211c601a3e0875541c1035ab408fbceaddf50", size = 88189, upload-time = "2025-11-03T13:01:52.605Z" }, + { url = "https://files.pythonhosted.org/packages/a5/0b/a54cd7406995ab097fceb907c7eb13a6ddd49e0b231e448f1a81a50af65c/lz4-4.4.5-cp312-cp312-win_amd64.whl", hash = "sha256:713a777de88a73425cf08eb11f742cd2c98628e79a8673d6a52e3c5f0c116f33", size = 99497, upload-time = "2025-11-03T13:01:53.477Z" }, + { url = "https://files.pythonhosted.org/packages/6a/7e/dc28a952e4bfa32ca16fa2eb026e7a6ce5d1411fcd5986cd08c74ec187b9/lz4-4.4.5-cp312-cp312-win_arm64.whl", hash = "sha256:a88cbb729cc333334ccfb52f070463c21560fca63afcf636a9f160a55fac3301", size = 91279, upload-time = "2025-11-03T13:01:54.419Z" }, + { url = "https://files.pythonhosted.org/packages/2f/46/08fd8ef19b782f301d56a9ccfd7dafec5fd4fc1a9f017cf22a1accb585d7/lz4-4.4.5-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:6bb05416444fafea170b07181bc70640975ecc2a8c92b3b658c554119519716c", size = 207171, upload-time = "2025-11-03T13:01:56.595Z" }, + { url = "https://files.pythonhosted.org/packages/8f/3f/ea3334e59de30871d773963997ecdba96c4584c5f8007fd83cfc8f1ee935/lz4-4.4.5-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:b424df1076e40d4e884cfcc4c77d815368b7fb9ebcd7e634f937725cd9a8a72a", size = 207163, upload-time = "2025-11-03T13:01:57.721Z" }, + { url = "https://files.pythonhosted.org/packages/41/7b/7b3a2a0feb998969f4793c650bb16eff5b06e80d1f7bff867feb332f2af2/lz4-4.4.5-cp313-cp313-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:216ca0c6c90719731c64f41cfbd6f27a736d7e50a10b70fad2a9c9b262ec923d", size = 1292136, upload-time = "2025-11-03T13:02:00.375Z" }, + { url = "https://files.pythonhosted.org/packages/89/d1/f1d259352227bb1c185288dd694121ea303e43404aa77560b879c90e7073/lz4-4.4.5-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:533298d208b58b651662dd972f52d807d48915176e5b032fb4f8c3b6f5fe535c", size = 1279639, upload-time = "2025-11-03T13:02:01.649Z" }, + { url = "https://files.pythonhosted.org/packages/d2/fb/ba9256c48266a09012ed1d9b0253b9aa4fe9cdff094f8febf5b26a4aa2a2/lz4-4.4.5-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:451039b609b9a88a934800b5fc6ee401c89ad9c175abf2f4d9f8b2e4ef1afc64", size = 1368257, upload-time = "2025-11-03T13:02:03.35Z" }, + { url = "https://files.pythonhosted.org/packages/a5/6d/dee32a9430c8b0e01bbb4537573cabd00555827f1a0a42d4e24ca803935c/lz4-4.4.5-cp313-cp313-win32.whl", hash = "sha256:a5f197ffa6fc0e93207b0af71b302e0a2f6f29982e5de0fbda61606dd3a55832", size = 88191, upload-time = "2025-11-03T13:02:04.406Z" }, + { url = "https://files.pythonhosted.org/packages/18/e0/f06028aea741bbecb2a7e9648f4643235279a770c7ffaf70bd4860c73661/lz4-4.4.5-cp313-cp313-win_amd64.whl", hash = "sha256:da68497f78953017deb20edff0dba95641cc86e7423dfadf7c0264e1ac60dc22", size = 99502, upload-time = "2025-11-03T13:02:05.886Z" }, + { url = "https://files.pythonhosted.org/packages/61/72/5bef44afb303e56078676b9f2486f13173a3c1e7f17eaac1793538174817/lz4-4.4.5-cp313-cp313-win_arm64.whl", hash = "sha256:c1cfa663468a189dab510ab231aad030970593f997746d7a324d40104db0d0a9", size = 91285, upload-time = "2025-11-03T13:02:06.77Z" }, + { url = "https://files.pythonhosted.org/packages/49/55/6a5c2952971af73f15ed4ebfdd69774b454bd0dc905b289082ca8664fba1/lz4-4.4.5-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:67531da3b62f49c939e09d56492baf397175ff39926d0bd5bd2d191ac2bff95f", size = 207348, upload-time = "2025-11-03T13:02:08.117Z" }, + { url = "https://files.pythonhosted.org/packages/4e/d7/fd62cbdbdccc35341e83aabdb3f6d5c19be2687d0a4eaf6457ddf53bba64/lz4-4.4.5-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:a1acbbba9edbcbb982bc2cac5e7108f0f553aebac1040fbec67a011a45afa1ba", size = 207340, upload-time = "2025-11-03T13:02:09.152Z" }, + { url = "https://files.pythonhosted.org/packages/77/69/225ffadaacb4b0e0eb5fd263541edd938f16cd21fe1eae3cd6d5b6a259dc/lz4-4.4.5-cp313-cp313t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:a482eecc0b7829c89b498fda883dbd50e98153a116de612ee7c111c8bcf82d1d", size = 1293398, upload-time = "2025-11-03T13:02:10.272Z" }, + { url = "https://files.pythonhosted.org/packages/c6/9e/2ce59ba4a21ea5dc43460cba6f34584e187328019abc0e66698f2b66c881/lz4-4.4.5-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e099ddfaa88f59dd8d36c8a3c66bd982b4984edf127eb18e30bb49bdba68ce67", size = 1281209, upload-time = "2025-11-03T13:02:12.091Z" }, + { url = "https://files.pythonhosted.org/packages/80/4f/4d946bd1624ec229b386a3bc8e7a85fa9a963d67d0a62043f0af0978d3da/lz4-4.4.5-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a2af2897333b421360fdcce895c6f6281dc3fab018d19d341cf64d043fc8d90d", size = 1369406, upload-time = "2025-11-03T13:02:13.683Z" }, + { url = "https://files.pythonhosted.org/packages/02/a2/d429ba4720a9064722698b4b754fb93e42e625f1318b8fe834086c7c783b/lz4-4.4.5-cp313-cp313t-win32.whl", hash = "sha256:66c5de72bf4988e1b284ebdd6524c4bead2c507a2d7f172201572bac6f593901", size = 88325, upload-time = "2025-11-03T13:02:14.743Z" }, + { url = "https://files.pythonhosted.org/packages/4b/85/7ba10c9b97c06af6c8f7032ec942ff127558863df52d866019ce9d2425cf/lz4-4.4.5-cp313-cp313t-win_amd64.whl", hash = "sha256:cdd4bdcbaf35056086d910d219106f6a04e1ab0daa40ec0eeef1626c27d0fddb", size = 99643, upload-time = "2025-11-03T13:02:15.978Z" }, + { url = "https://files.pythonhosted.org/packages/77/4d/a175459fb29f909e13e57c8f475181ad8085d8d7869bd8ad99033e3ee5fa/lz4-4.4.5-cp313-cp313t-win_arm64.whl", hash = "sha256:28ccaeb7c5222454cd5f60fcd152564205bcb801bd80e125949d2dfbadc76bbd", size = 91504, upload-time = "2025-11-03T13:02:17.313Z" }, + { url = "https://files.pythonhosted.org/packages/63/9c/70bdbdb9f54053a308b200b4678afd13efd0eafb6ddcbb7f00077213c2e5/lz4-4.4.5-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:c216b6d5275fc060c6280936bb3bb0e0be6126afb08abccde27eed23dead135f", size = 207586, upload-time = "2025-11-03T13:02:18.263Z" }, + { url = "https://files.pythonhosted.org/packages/b6/cb/bfead8f437741ce51e14b3c7d404e3a1f6b409c440bad9b8f3945d4c40a7/lz4-4.4.5-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c8e71b14938082ebaf78144f3b3917ac715f72d14c076f384a4c062df96f9df6", size = 207161, upload-time = "2025-11-03T13:02:19.286Z" }, + { url = "https://files.pythonhosted.org/packages/e7/18/b192b2ce465dfbeabc4fc957ece7a1d34aded0d95a588862f1c8a86ac448/lz4-4.4.5-cp314-cp314-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:9b5e6abca8df9f9bdc5c3085f33ff32cdc86ed04c65e0355506d46a5ac19b6e9", size = 1292415, upload-time = "2025-11-03T13:02:20.829Z" }, + { url = "https://files.pythonhosted.org/packages/67/79/a4e91872ab60f5e89bfad3e996ea7dc74a30f27253faf95865771225ccba/lz4-4.4.5-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3b84a42da86e8ad8537aabef062e7f661f4a877d1c74d65606c49d835d36d668", size = 1279920, upload-time = "2025-11-03T13:02:22.013Z" }, + { url = "https://files.pythonhosted.org/packages/f1/01/d52c7b11eaa286d49dae619c0eec4aabc0bf3cda7a7467eb77c62c4471f3/lz4-4.4.5-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0bba042ec5a61fa77c7e380351a61cb768277801240249841defd2ff0a10742f", size = 1368661, upload-time = "2025-11-03T13:02:23.208Z" }, + { url = "https://files.pythonhosted.org/packages/f7/da/137ddeea14c2cb86864838277b2607d09f8253f152156a07f84e11768a28/lz4-4.4.5-cp314-cp314-win32.whl", hash = "sha256:bd85d118316b53ed73956435bee1997bd06cc66dd2fa74073e3b1322bd520a67", size = 90139, upload-time = "2025-11-03T13:02:24.301Z" }, + { url = "https://files.pythonhosted.org/packages/18/2c/8332080fd293f8337779a440b3a143f85e374311705d243439a3349b81ad/lz4-4.4.5-cp314-cp314-win_amd64.whl", hash = "sha256:92159782a4502858a21e0079d77cdcaade23e8a5d252ddf46b0652604300d7be", size = 101497, upload-time = "2025-11-03T13:02:25.187Z" }, + { url = "https://files.pythonhosted.org/packages/ca/28/2635a8141c9a4f4bc23f5135a92bbcf48d928d8ca094088c962df1879d64/lz4-4.4.5-cp314-cp314-win_arm64.whl", hash = "sha256:d994b87abaa7a88ceb7a37c90f547b8284ff9da694e6afcfaa8568d739faf3f7", size = 93812, upload-time = "2025-11-03T13:02:26.133Z" }, +] + [[package]] name = "markdown-it-py" version = "4.2.0" @@ -2711,6 +2777,32 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/41/09/5b161152e2d90f7b87f781c2e1267494aef9c32498df793f73ad0a0a494a/matplotlib_inline-0.2.2-py3-none-any.whl", hash = "sha256:3c821cf1c209f59fb2d2d64abbf5b23b67bcb2210d663f9918dd851c6da1fcf6", size = 9534, upload-time = "2026-05-08T17:33:32.055Z" }, ] +[[package]] +name = "mcap" +version = "1.5.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "lz4" }, + { name = "zstandard" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/26/16/2ac1cbeaa524453d616050db9fda14010eda507fd8130e0676879bb8faf9/mcap-1.5.0.tar.gz", hash = "sha256:9c385cc5e5a6bccff4aa0c6814b305faa2c3239ece46778661eaa6d809b1768a", size = 23110, upload-time = "2026-09-24T21:45:44.714Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/13/59/caa4d60a58f129659ea5403e6fb52808f87005dfcbaa03e84f0e07f0033e/mcap-1.5.0-py3-none-any.whl", hash = "sha256:44ba129d381abdca474fbf5bcee036db87d7075d9059b139bf2fee2975c8bd52", size = 21021, upload-time = "2026-09-24T21:45:43.51Z" }, +] + +[[package]] +name = "mcap-protobuf-support" +version = "0.5.4" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "mcap" }, + { name = "protobuf" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/cc/8e/98af824adce25082f1c7fb8e8aab48b1678a370f97633a55bfb06c40e3fd/mcap_protobuf_support-0.5.4.tar.gz", hash = "sha256:3af2de2c2dbda9d1dbb1a526d2ad0cf3604ab44947dd1c99a52a2f990b1eb308", size = 7059, upload-time = "2025-12-24T21:22:57.716Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/74/ee/cb4beace2bbaf3877848dbc555366c5a72ec1dabba5d620bf746143d91c4/mcap_protobuf_support-0.5.4-py3-none-any.whl", hash = "sha256:072f83b7b147b5faa25df21c175cbf6e53bf959f8a6aea534167055bc4526649", size = 7286, upload-time = "2025-12-24T21:22:56.393Z" }, +] + [[package]] name = "mdurl" version = "0.1.2" @@ -3309,7 +3401,7 @@ name = "nvidia-cudnn-cu12" version = "9.5.1.17" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-cublas-cu12", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" }, + { name = "nvidia-cublas-cu12" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/2a/78/4535c9c7f859a64781e43c969a3a7e84c54634e319a996d43ef32ce46f83/nvidia_cudnn_cu12-9.5.1.17-py3-none-manylinux_2_28_x86_64.whl", hash = "sha256:30ac3869f6db17d170e0e556dd6cc5eee02647abc31ca856634d5a40f82c15b2", size = 570988386, upload-time = "2024-10-25T19:54:26.39Z" }, @@ -3320,7 +3412,7 @@ name = "nvidia-cufft-cu12" version = "11.3.0.4" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-nvjitlink-cu12", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" }, + { name = "nvidia-nvjitlink-cu12" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/8f/16/73727675941ab8e6ffd86ca3a4b7b47065edcca7a997920b831f8147c99d/nvidia_cufft_cu12-11.3.0.4-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:ccba62eb9cef5559abd5e0d54ceed2d9934030f51163df018532142a8ec533e5", size = 200221632, upload-time = "2024-11-20T17:41:32.357Z" }, @@ -3349,9 +3441,9 @@ name = "nvidia-cusolver-cu12" version = "11.7.1.2" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-cublas-cu12", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" }, - { name = "nvidia-cusparse-cu12", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" }, - { name = "nvidia-nvjitlink-cu12", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" }, + { name = "nvidia-cublas-cu12" }, + { name = "nvidia-cusparse-cu12" }, + { name = "nvidia-nvjitlink-cu12" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/f0/6e/c2cf12c9ff8b872e92b4a5740701e51ff17689c4d726fca91875b07f655d/nvidia_cusolver_cu12-11.7.1.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:e9e49843a7707e42022babb9bcfa33c29857a93b88020c4e4434656a655b698c", size = 158229790, upload-time = "2024-11-20T17:43:43.211Z" }, @@ -3363,7 +3455,7 @@ name = "nvidia-cusparse-cu12" version = "12.5.4.2" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-nvjitlink-cu12", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" }, + { name = "nvidia-nvjitlink-cu12" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/06/1e/b8b7c2f4099a37b96af5c9bb158632ea9e5d9d27d7391d7eb8fc45236674/nvidia_cusparse_cu12-12.5.4.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7556d9eca156e18184b94947ade0fba5bb47d69cec46bf8660fd2c71a4b48b73", size = 216561367, upload-time = "2024-11-20T17:44:54.824Z" }, @@ -3572,7 +3664,7 @@ name = "pexpect" version = "4.9.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "ptyprocess", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" }, + { name = "ptyprocess" }, ] sdist = { url = "https://files.pythonhosted.org/packages/42/92/cc564bf6381ff43ce1f4d06852fc19a2f11d180f23dc32d9588bee2f149d/pexpect-4.9.0.tar.gz", hash = "sha256:ee7d41123f3c9911050ea2c2dac107568dc43b2d3b0c7557a33212c398ead30f", size = 166450, upload-time = "2023-11-25T09:07:26.339Z" } wheels = [ @@ -3707,7 +3799,7 @@ resolution-markers = [ "python_full_version < '3.12' and sys_platform != 'emscripten' and sys_platform != 'win32'", ] dependencies = [ - { name = "numpy", marker = "python_full_version < '3.12'" }, + { name = "numpy" }, ] sdist = { url = "https://files.pythonhosted.org/packages/53/ae/677391c5e75858b5496dcdf4146952646c13a8b31c04090fa98c37a04198/positional_encodings-6.0.3.tar.gz", hash = "sha256:c144ad52e78f09d16effcbfabdf3e84050b56a37e342248476e18bbffd747b24", size = 6626, upload-time = "2024-01-04T18:17:44.133Z" } wheels = [ @@ -3716,7 +3808,7 @@ wheels = [ [package.optional-dependencies] pytorch = [ - { name = "torch", marker = "python_full_version < '3.12'" }, + { name = "torch" }, ] [[package]] @@ -3728,14 +3820,14 @@ resolution-markers = [ "python_full_version >= '3.14' and sys_platform == 'emscripten'", "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'", "python_full_version == '3.13.*' and sys_platform == 'win32'", - "python_full_version == '3.12.*' and sys_platform == 'win32'", "python_full_version == '3.13.*' and sys_platform == 'emscripten'", - "python_full_version == '3.12.*' and sys_platform == 'emscripten'", "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.12.*' and sys_platform == 'win32'", + "python_full_version == '3.12.*' and sys_platform == 'emscripten'", "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", ] dependencies = [ - { name = "numpy", marker = "python_full_version >= '3.12'" }, + { name = "numpy" }, ] sdist = { url = "https://files.pythonhosted.org/packages/6d/04/93a78a33088fe9115602883a39f8395cdb5561388a24e65f57c11920c638/positional_encodings-6.0.4.tar.gz", hash = "sha256:435db9596a73759e7caa8f677ebf6d2ee4d39187112244e4d8b6d7b62bc119ad", size = 6814, upload-time = "2024-10-23T07:33:48.273Z" } wheels = [ @@ -3744,7 +3836,7 @@ wheels = [ [package.optional-dependencies] pytorch = [ - { name = "torch", marker = "python_full_version >= '3.12'" }, + { name = "torch" }, ] [[package]] @@ -5641,7 +5733,7 @@ name = "triton" version = "3.3.1" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "setuptools", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" }, + { name = "setuptools" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/21/2f/3e56ea7b58f80ff68899b1dbe810ff257c9d177d288c6b0f55bf2fe4eb50/triton-3.3.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b31e3aa26f8cb3cc5bf4e187bf737cbacf17311e1112b781d4a059353dfd731b", size = 155689937, upload-time = "2025-05-29T23:39:44.182Z" }, @@ -6163,3 +6255,77 @@ sdist = { url = "https://files.pythonhosted.org/packages/30/21/093488dfc7cc8964d wheels = [ { url = "https://files.pythonhosted.org/packages/08/8a/0861bec20485572fbddf3dfba2910e38fe249796cb73ecdeb74e07eeb8d3/zipp-3.23.1-py3-none-any.whl", hash = "sha256:0b3596c50a5c700c9cb40ba8d86d9f2cc4807e9bedb06bcdf7fac85633e444dc", size = 10378, upload-time = "2026-04-13T23:21:45.386Z" }, ] + +[[package]] +name = "zstandard" +version = "0.25.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/fd/aa/3e0508d5a5dd96529cdc5a97011299056e14c6505b678fd58938792794b1/zstandard-0.25.0.tar.gz", hash = "sha256:7713e1179d162cf5c7906da876ec2ccb9c3a9dcbdffef0cc7f70c3667a205f0b", size = 711513, upload-time = "2025-09-14T22:15:54.002Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/2a/83/c3ca27c363d104980f1c9cee1101cc8ba724ac8c28a033ede6aab89585b1/zstandard-0.25.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:933b65d7680ea337180733cf9e87293cc5500cc0eb3fc8769f4d3c88d724ec5c", size = 795254, upload-time = "2025-09-14T22:16:26.137Z" }, + { url = "https://files.pythonhosted.org/packages/ac/4d/e66465c5411a7cf4866aeadc7d108081d8ceba9bc7abe6b14aa21c671ec3/zstandard-0.25.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:a3f79487c687b1fc69f19e487cd949bf3aae653d181dfb5fde3bf6d18894706f", size = 640559, upload-time = "2025-09-14T22:16:27.973Z" }, + { url = "https://files.pythonhosted.org/packages/12/56/354fe655905f290d3b147b33fe946b0f27e791e4b50a5f004c802cb3eb7b/zstandard-0.25.0-cp311-cp311-manylinux2010_i686.manylinux2014_i686.manylinux_2_12_i686.manylinux_2_17_i686.whl", hash = "sha256:0bbc9a0c65ce0eea3c34a691e3c4b6889f5f3909ba4822ab385fab9057099431", size = 5348020, upload-time = "2025-09-14T22:16:29.523Z" }, + { url = "https://files.pythonhosted.org/packages/3b/13/2b7ed68bd85e69a2069bcc72141d378f22cae5a0f3b353a2c8f50ef30c1b/zstandard-0.25.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:01582723b3ccd6939ab7b3a78622c573799d5d8737b534b86d0e06ac18dbde4a", size = 5058126, upload-time = "2025-09-14T22:16:31.811Z" }, + { url = "https://files.pythonhosted.org/packages/c9/dd/fdaf0674f4b10d92cb120ccff58bbb6626bf8368f00ebfd2a41ba4a0dc99/zstandard-0.25.0-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:5f1ad7bf88535edcf30038f6919abe087f606f62c00a87d7e33e7fc57cb69fcc", size = 5405390, upload-time = "2025-09-14T22:16:33.486Z" }, + { url = "https://files.pythonhosted.org/packages/0f/67/354d1555575bc2490435f90d67ca4dd65238ff2f119f30f72d5cde09c2ad/zstandard-0.25.0-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:06acb75eebeedb77b69048031282737717a63e71e4ae3f77cc0c3b9508320df6", size = 5452914, upload-time = "2025-09-14T22:16:35.277Z" }, + { url = "https://files.pythonhosted.org/packages/bb/1f/e9cfd801a3f9190bf3e759c422bbfd2247db9d7f3d54a56ecde70137791a/zstandard-0.25.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:9300d02ea7c6506f00e627e287e0492a5eb0371ec1670ae852fefffa6164b072", size = 5559635, upload-time = "2025-09-14T22:16:37.141Z" }, + { url = "https://files.pythonhosted.org/packages/21/88/5ba550f797ca953a52d708c8e4f380959e7e3280af029e38fbf47b55916e/zstandard-0.25.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:bfd06b1c5584b657a2892a6014c2f4c20e0db0208c159148fa78c65f7e0b0277", size = 5048277, upload-time = "2025-09-14T22:16:38.807Z" }, + { url = "https://files.pythonhosted.org/packages/46/c0/ca3e533b4fa03112facbe7fbe7779cb1ebec215688e5df576fe5429172e0/zstandard-0.25.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:f373da2c1757bb7f1acaf09369cdc1d51d84131e50d5fa9863982fd626466313", size = 5574377, upload-time = "2025-09-14T22:16:40.523Z" }, + { url = "https://files.pythonhosted.org/packages/12/9b/3fb626390113f272abd0799fd677ea33d5fc3ec185e62e6be534493c4b60/zstandard-0.25.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:6c0e5a65158a7946e7a7affa6418878ef97ab66636f13353b8502d7ea03c8097", size = 4961493, upload-time = "2025-09-14T22:16:43.3Z" }, + { url = "https://files.pythonhosted.org/packages/cb/d3/23094a6b6a4b1343b27ae68249daa17ae0651fcfec9ed4de09d14b940285/zstandard-0.25.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:c8e167d5adf59476fa3e37bee730890e389410c354771a62e3c076c86f9f7778", size = 5269018, upload-time = "2025-09-14T22:16:45.292Z" }, + { url = "https://files.pythonhosted.org/packages/8c/a7/bb5a0c1c0f3f4b5e9d5b55198e39de91e04ba7c205cc46fcb0f95f0383c1/zstandard-0.25.0-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:98750a309eb2f020da61e727de7d7ba3c57c97cf6213f6f6277bb7fb42a8e065", size = 5443672, upload-time = "2025-09-14T22:16:47.076Z" }, + { url = "https://files.pythonhosted.org/packages/27/22/503347aa08d073993f25109c36c8d9f029c7d5949198050962cb568dfa5e/zstandard-0.25.0-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:22a086cff1b6ceca18a8dd6096ec631e430e93a8e70a9ca5efa7561a00f826fa", size = 5822753, upload-time = "2025-09-14T22:16:49.316Z" }, + { url = "https://files.pythonhosted.org/packages/e2/be/94267dc6ee64f0f8ba2b2ae7c7a2df934a816baaa7291db9e1aa77394c3c/zstandard-0.25.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:72d35d7aa0bba323965da807a462b0966c91608ef3a48ba761678cb20ce5d8b7", size = 5366047, upload-time = "2025-09-14T22:16:51.328Z" }, + { url = "https://files.pythonhosted.org/packages/7b/a3/732893eab0a3a7aecff8b99052fecf9f605cf0fb5fb6d0290e36beee47a4/zstandard-0.25.0-cp311-cp311-win32.whl", hash = "sha256:f5aeea11ded7320a84dcdd62a3d95b5186834224a9e55b92ccae35d21a8b63d4", size = 436484, upload-time = "2025-09-14T22:16:55.005Z" }, + { url = "https://files.pythonhosted.org/packages/43/a3/c6155f5c1cce691cb80dfd38627046e50af3ee9ddc5d0b45b9b063bfb8c9/zstandard-0.25.0-cp311-cp311-win_amd64.whl", hash = "sha256:daab68faadb847063d0c56f361a289c4f268706b598afbf9ad113cbe5c38b6b2", size = 506183, upload-time = "2025-09-14T22:16:52.753Z" }, + { url = "https://files.pythonhosted.org/packages/8c/3e/8945ab86a0820cc0e0cdbf38086a92868a9172020fdab8a03ac19662b0e5/zstandard-0.25.0-cp311-cp311-win_arm64.whl", hash = "sha256:22a06c5df3751bb7dc67406f5374734ccee8ed37fc5981bf1ad7041831fa1137", size = 462533, upload-time = "2025-09-14T22:16:53.878Z" }, + { url = "https://files.pythonhosted.org/packages/82/fc/f26eb6ef91ae723a03e16eddb198abcfce2bc5a42e224d44cc8b6765e57e/zstandard-0.25.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7b3c3a3ab9daa3eed242d6ecceead93aebbb8f5f84318d82cee643e019c4b73b", size = 795738, upload-time = "2025-09-14T22:16:56.237Z" }, + { url = "https://files.pythonhosted.org/packages/aa/1c/d920d64b22f8dd028a8b90e2d756e431a5d86194caa78e3819c7bf53b4b3/zstandard-0.25.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:913cbd31a400febff93b564a23e17c3ed2d56c064006f54efec210d586171c00", size = 640436, upload-time = "2025-09-14T22:16:57.774Z" }, + { url = "https://files.pythonhosted.org/packages/53/6c/288c3f0bd9fcfe9ca41e2c2fbfd17b2097f6af57b62a81161941f09afa76/zstandard-0.25.0-cp312-cp312-manylinux2010_i686.manylinux2014_i686.manylinux_2_12_i686.manylinux_2_17_i686.whl", hash = "sha256:011d388c76b11a0c165374ce660ce2c8efa8e5d87f34996aa80f9c0816698b64", size = 5343019, upload-time = "2025-09-14T22:16:59.302Z" }, + { url = "https://files.pythonhosted.org/packages/1e/15/efef5a2f204a64bdb5571e6161d49f7ef0fffdbca953a615efbec045f60f/zstandard-0.25.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:6dffecc361d079bb48d7caef5d673c88c8988d3d33fb74ab95b7ee6da42652ea", size = 5063012, upload-time = "2025-09-14T22:17:01.156Z" }, + { url = "https://files.pythonhosted.org/packages/b7/37/a6ce629ffdb43959e92e87ebdaeebb5ac81c944b6a75c9c47e300f85abdf/zstandard-0.25.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:7149623bba7fdf7e7f24312953bcf73cae103db8cae49f8154dd1eadc8a29ecb", size = 5394148, upload-time = "2025-09-14T22:17:03.091Z" }, + { url = "https://files.pythonhosted.org/packages/e3/79/2bf870b3abeb5c070fe2d670a5a8d1057a8270f125ef7676d29ea900f496/zstandard-0.25.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:6a573a35693e03cf1d67799fd01b50ff578515a8aeadd4595d2a7fa9f3ec002a", size = 5451652, upload-time = "2025-09-14T22:17:04.979Z" }, + { url = "https://files.pythonhosted.org/packages/53/60/7be26e610767316c028a2cbedb9a3beabdbe33e2182c373f71a1c0b88f36/zstandard-0.25.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:5a56ba0db2d244117ed744dfa8f6f5b366e14148e00de44723413b2f3938a902", size = 5546993, upload-time = "2025-09-14T22:17:06.781Z" }, + { url = "https://files.pythonhosted.org/packages/85/c7/3483ad9ff0662623f3648479b0380d2de5510abf00990468c286c6b04017/zstandard-0.25.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:10ef2a79ab8e2974e2075fb984e5b9806c64134810fac21576f0668e7ea19f8f", size = 5046806, upload-time = "2025-09-14T22:17:08.415Z" }, + { url = "https://files.pythonhosted.org/packages/08/b3/206883dd25b8d1591a1caa44b54c2aad84badccf2f1de9e2d60a446f9a25/zstandard-0.25.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:aaf21ba8fb76d102b696781bddaa0954b782536446083ae3fdaa6f16b25a1c4b", size = 5576659, upload-time = "2025-09-14T22:17:10.164Z" }, + { url = "https://files.pythonhosted.org/packages/9d/31/76c0779101453e6c117b0ff22565865c54f48f8bd807df2b00c2c404b8e0/zstandard-0.25.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:1869da9571d5e94a85a5e8d57e4e8807b175c9e4a6294e3b66fa4efb074d90f6", size = 4953933, upload-time = "2025-09-14T22:17:11.857Z" }, + { url = "https://files.pythonhosted.org/packages/18/e1/97680c664a1bf9a247a280a053d98e251424af51f1b196c6d52f117c9720/zstandard-0.25.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:809c5bcb2c67cd0ed81e9229d227d4ca28f82d0f778fc5fea624a9def3963f91", size = 5268008, upload-time = "2025-09-14T22:17:13.627Z" }, + { url = "https://files.pythonhosted.org/packages/1e/73/316e4010de585ac798e154e88fd81bb16afc5c5cb1a72eeb16dd37e8024a/zstandard-0.25.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:f27662e4f7dbf9f9c12391cb37b4c4c3cb90ffbd3b1fb9284dadbbb8935fa708", size = 5433517, upload-time = "2025-09-14T22:17:16.103Z" }, + { url = "https://files.pythonhosted.org/packages/5b/60/dd0f8cfa8129c5a0ce3ea6b7f70be5b33d2618013a161e1ff26c2b39787c/zstandard-0.25.0-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:99c0c846e6e61718715a3c9437ccc625de26593fea60189567f0118dc9db7512", size = 5814292, upload-time = "2025-09-14T22:17:17.827Z" }, + { url = "https://files.pythonhosted.org/packages/fc/5f/75aafd4b9d11b5407b641b8e41a57864097663699f23e9ad4dbb91dc6bfe/zstandard-0.25.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:474d2596a2dbc241a556e965fb76002c1ce655445e4e3bf38e5477d413165ffa", size = 5360237, upload-time = "2025-09-14T22:17:19.954Z" }, + { url = "https://files.pythonhosted.org/packages/ff/8d/0309daffea4fcac7981021dbf21cdb2e3427a9e76bafbcdbdf5392ff99a4/zstandard-0.25.0-cp312-cp312-win32.whl", hash = "sha256:23ebc8f17a03133b4426bcc04aabd68f8236eb78c3760f12783385171b0fd8bd", size = 436922, upload-time = "2025-09-14T22:17:24.398Z" }, + { url = "https://files.pythonhosted.org/packages/79/3b/fa54d9015f945330510cb5d0b0501e8253c127cca7ebe8ba46a965df18c5/zstandard-0.25.0-cp312-cp312-win_amd64.whl", hash = "sha256:ffef5a74088f1e09947aecf91011136665152e0b4b359c42be3373897fb39b01", size = 506276, upload-time = "2025-09-14T22:17:21.429Z" }, + { url = "https://files.pythonhosted.org/packages/ea/6b/8b51697e5319b1f9ac71087b0af9a40d8a6288ff8025c36486e0c12abcc4/zstandard-0.25.0-cp312-cp312-win_arm64.whl", hash = "sha256:181eb40e0b6a29b3cd2849f825e0fa34397f649170673d385f3598ae17cca2e9", size = 462679, upload-time = "2025-09-14T22:17:23.147Z" }, + { url = "https://files.pythonhosted.org/packages/35/0b/8df9c4ad06af91d39e94fa96cc010a24ac4ef1378d3efab9223cc8593d40/zstandard-0.25.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:ec996f12524f88e151c339688c3897194821d7f03081ab35d31d1e12ec975e94", size = 795735, upload-time = "2025-09-14T22:17:26.042Z" }, + { url = "https://files.pythonhosted.org/packages/3f/06/9ae96a3e5dcfd119377ba33d4c42a7d89da1efabd5cb3e366b156c45ff4d/zstandard-0.25.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:a1a4ae2dec3993a32247995bdfe367fc3266da832d82f8438c8570f989753de1", size = 640440, upload-time = "2025-09-14T22:17:27.366Z" }, + { url = "https://files.pythonhosted.org/packages/d9/14/933d27204c2bd404229c69f445862454dcc101cd69ef8c6068f15aaec12c/zstandard-0.25.0-cp313-cp313-manylinux2010_i686.manylinux2014_i686.manylinux_2_12_i686.manylinux_2_17_i686.whl", hash = "sha256:e96594a5537722fdfb79951672a2a63aec5ebfb823e7560586f7484819f2a08f", size = 5343070, upload-time = "2025-09-14T22:17:28.896Z" }, + { url = "https://files.pythonhosted.org/packages/6d/db/ddb11011826ed7db9d0e485d13df79b58586bfdec56e5c84a928a9a78c1c/zstandard-0.25.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:bfc4e20784722098822e3eee42b8e576b379ed72cca4a7cb856ae733e62192ea", size = 5063001, upload-time = "2025-09-14T22:17:31.044Z" }, + { url = "https://files.pythonhosted.org/packages/db/00/87466ea3f99599d02a5238498b87bf84a6348290c19571051839ca943777/zstandard-0.25.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:457ed498fc58cdc12fc48f7950e02740d4f7ae9493dd4ab2168a47c93c31298e", size = 5394120, upload-time = "2025-09-14T22:17:32.711Z" }, + { url = "https://files.pythonhosted.org/packages/2b/95/fc5531d9c618a679a20ff6c29e2b3ef1d1f4ad66c5e161ae6ff847d102a9/zstandard-0.25.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:fd7a5004eb1980d3cefe26b2685bcb0b17989901a70a1040d1ac86f1d898c551", size = 5451230, upload-time = "2025-09-14T22:17:34.41Z" }, + { url = "https://files.pythonhosted.org/packages/63/4b/e3678b4e776db00f9f7b2fe58e547e8928ef32727d7a1ff01dea010f3f13/zstandard-0.25.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:8e735494da3db08694d26480f1493ad2cf86e99bdd53e8e9771b2752a5c0246a", size = 5547173, upload-time = "2025-09-14T22:17:36.084Z" }, + { url = "https://files.pythonhosted.org/packages/4e/d5/ba05ed95c6b8ec30bd468dfeab20589f2cf709b5c940483e31d991f2ca58/zstandard-0.25.0-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:3a39c94ad7866160a4a46d772e43311a743c316942037671beb264e395bdd611", size = 5046736, upload-time = "2025-09-14T22:17:37.891Z" }, + { url = "https://files.pythonhosted.org/packages/50/d5/870aa06b3a76c73eced65c044b92286a3c4e00554005ff51962deef28e28/zstandard-0.25.0-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:172de1f06947577d3a3005416977cce6168f2261284c02080e7ad0185faeced3", size = 5576368, upload-time = "2025-09-14T22:17:40.206Z" }, + { url = "https://files.pythonhosted.org/packages/5d/35/398dc2ffc89d304d59bc12f0fdd931b4ce455bddf7038a0a67733a25f550/zstandard-0.25.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:3c83b0188c852a47cd13ef3bf9209fb0a77fa5374958b8c53aaa699398c6bd7b", size = 4954022, upload-time = "2025-09-14T22:17:41.879Z" }, + { url = "https://files.pythonhosted.org/packages/9a/5c/36ba1e5507d56d2213202ec2b05e8541734af5f2ce378c5d1ceaf4d88dc4/zstandard-0.25.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:1673b7199bbe763365b81a4f3252b8e80f44c9e323fc42940dc8843bfeaf9851", size = 5267889, upload-time = "2025-09-14T22:17:43.577Z" }, + { url = "https://files.pythonhosted.org/packages/70/e8/2ec6b6fb7358b2ec0113ae202647ca7c0e9d15b61c005ae5225ad0995df5/zstandard-0.25.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:0be7622c37c183406f3dbf0cba104118eb16a4ea7359eeb5752f0794882fc250", size = 5433952, upload-time = "2025-09-14T22:17:45.271Z" }, + { url = "https://files.pythonhosted.org/packages/7b/01/b5f4d4dbc59ef193e870495c6f1275f5b2928e01ff5a81fecb22a06e22fb/zstandard-0.25.0-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:5f5e4c2a23ca271c218ac025bd7d635597048b366d6f31f420aaeb715239fc98", size = 5814054, upload-time = "2025-09-14T22:17:47.08Z" }, + { url = "https://files.pythonhosted.org/packages/b2/e5/fbd822d5c6f427cf158316d012c5a12f233473c2f9c5fe5ab1ae5d21f3d8/zstandard-0.25.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:4f187a0bb61b35119d1926aee039524d1f93aaf38a9916b8c4b78ac8514a0aaf", size = 5360113, upload-time = "2025-09-14T22:17:48.893Z" }, + { url = "https://files.pythonhosted.org/packages/8e/e0/69a553d2047f9a2c7347caa225bb3a63b6d7704ad74610cb7823baa08ed7/zstandard-0.25.0-cp313-cp313-win32.whl", hash = "sha256:7030defa83eef3e51ff26f0b7bfb229f0204b66fe18e04359ce3474ac33cbc09", size = 436936, upload-time = "2025-09-14T22:17:52.658Z" }, + { url = "https://files.pythonhosted.org/packages/d9/82/b9c06c870f3bd8767c201f1edbdf9e8dc34be5b0fbc5682c4f80fe948475/zstandard-0.25.0-cp313-cp313-win_amd64.whl", hash = "sha256:1f830a0dac88719af0ae43b8b2d6aef487d437036468ef3c2ea59c51f9d55fd5", size = 506232, upload-time = "2025-09-14T22:17:50.402Z" }, + { url = "https://files.pythonhosted.org/packages/d4/57/60c3c01243bb81d381c9916e2a6d9e149ab8627c0c7d7abb2d73384b3c0c/zstandard-0.25.0-cp313-cp313-win_arm64.whl", hash = "sha256:85304a43f4d513f5464ceb938aa02c1e78c2943b29f44a750b48b25ac999a049", size = 462671, upload-time = "2025-09-14T22:17:51.533Z" }, + { url = "https://files.pythonhosted.org/packages/3d/5c/f8923b595b55fe49e30612987ad8bf053aef555c14f05bb659dd5dbe3e8a/zstandard-0.25.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:e29f0cf06974c899b2c188ef7f783607dbef36da4c242eb6c82dcd8b512855e3", size = 795887, upload-time = "2025-09-14T22:17:54.198Z" }, + { url = "https://files.pythonhosted.org/packages/8d/09/d0a2a14fc3439c5f874042dca72a79c70a532090b7ba0003be73fee37ae2/zstandard-0.25.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:05df5136bc5a011f33cd25bc9f506e7426c0c9b3f9954f056831ce68f3b6689f", size = 640658, upload-time = "2025-09-14T22:17:55.423Z" }, + { url = "https://files.pythonhosted.org/packages/5d/7c/8b6b71b1ddd517f68ffb55e10834388d4f793c49c6b83effaaa05785b0b4/zstandard-0.25.0-cp314-cp314-manylinux2010_i686.manylinux_2_12_i686.manylinux_2_28_i686.whl", hash = "sha256:f604efd28f239cc21b3adb53eb061e2a205dc164be408e553b41ba2ffe0ca15c", size = 5379849, upload-time = "2025-09-14T22:17:57.372Z" }, + { url = "https://files.pythonhosted.org/packages/a4/86/a48e56320d0a17189ab7a42645387334fba2200e904ee47fc5a26c1fd8ca/zstandard-0.25.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:223415140608d0f0da010499eaa8ccdb9af210a543fac54bce15babbcfc78439", size = 5058095, upload-time = "2025-09-14T22:17:59.498Z" }, + { url = "https://files.pythonhosted.org/packages/f8/ad/eb659984ee2c0a779f9d06dbfe45e2dc39d99ff40a319895df2d3d9a48e5/zstandard-0.25.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:2e54296a283f3ab5a26fc9b8b5d4978ea0532f37b231644f367aa588930aa043", size = 5551751, upload-time = "2025-09-14T22:18:01.618Z" }, + { url = "https://files.pythonhosted.org/packages/61/b3/b637faea43677eb7bd42ab204dfb7053bd5c4582bfe6b1baefa80ac0c47b/zstandard-0.25.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:ca54090275939dc8ec5dea2d2afb400e0f83444b2fc24e07df7fdef677110859", size = 6364818, upload-time = "2025-09-14T22:18:03.769Z" }, + { url = "https://files.pythonhosted.org/packages/31/dc/cc50210e11e465c975462439a492516a73300ab8caa8f5e0902544fd748b/zstandard-0.25.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e09bb6252b6476d8d56100e8147b803befa9a12cea144bbe629dd508800d1ad0", size = 5560402, upload-time = "2025-09-14T22:18:05.954Z" }, + { url = "https://files.pythonhosted.org/packages/c9/ae/56523ae9c142f0c08efd5e868a6da613ae76614eca1305259c3bf6a0ed43/zstandard-0.25.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a9ec8c642d1ec73287ae3e726792dd86c96f5681eb8df274a757bf62b750eae7", size = 4955108, upload-time = "2025-09-14T22:18:07.68Z" }, + { url = "https://files.pythonhosted.org/packages/98/cf/c899f2d6df0840d5e384cf4c4121458c72802e8bda19691f3b16619f51e9/zstandard-0.25.0-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:a4089a10e598eae6393756b036e0f419e8c1d60f44a831520f9af41c14216cf2", size = 5269248, upload-time = "2025-09-14T22:18:09.753Z" }, + { url = "https://files.pythonhosted.org/packages/1b/c0/59e912a531d91e1c192d3085fc0f6fb2852753c301a812d856d857ea03c6/zstandard-0.25.0-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:f67e8f1a324a900e75b5e28ffb152bcac9fbed1cc7b43f99cd90f395c4375344", size = 5430330, upload-time = "2025-09-14T22:18:11.966Z" }, + { url = "https://files.pythonhosted.org/packages/a0/1d/7e31db1240de2df22a58e2ea9a93fc6e38cc29353e660c0272b6735d6669/zstandard-0.25.0-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:9654dbc012d8b06fc3d19cc825af3f7bf8ae242226df5f83936cb39f5fdc846c", size = 5811123, upload-time = "2025-09-14T22:18:13.907Z" }, + { url = "https://files.pythonhosted.org/packages/f6/49/fac46df5ad353d50535e118d6983069df68ca5908d4d65b8c466150a4ff1/zstandard-0.25.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:4203ce3b31aec23012d3a4cf4a2ed64d12fea5269c49aed5e4c3611b938e4088", size = 5359591, upload-time = "2025-09-14T22:18:16.465Z" }, + { url = "https://files.pythonhosted.org/packages/c2/38/f249a2050ad1eea0bb364046153942e34abba95dd5520af199aed86fbb49/zstandard-0.25.0-cp314-cp314-win32.whl", hash = "sha256:da469dc041701583e34de852d8634703550348d5822e66a0c827d39b05365b12", size = 444513, upload-time = "2025-09-14T22:18:20.61Z" }, + { url = "https://files.pythonhosted.org/packages/3a/43/241f9615bcf8ba8903b3f0432da069e857fc4fd1783bd26183db53c4804b/zstandard-0.25.0-cp314-cp314-win_amd64.whl", hash = "sha256:c19bcdd826e95671065f8692b5a4aa95c52dc7a02a4c5a0cac46deb879a017a2", size = 516118, upload-time = "2025-09-14T22:18:17.849Z" }, + { url = "https://files.pythonhosted.org/packages/f0/ef/da163ce2450ed4febf6467d77ccb4cd52c4c30ab45624bad26ca0a27260c/zstandard-0.25.0-cp314-cp314-win_arm64.whl", hash = "sha256:d7541afd73985c630bafcd6338d2518ae96060075f9463d7dc14cfb33514383d", size = 476940, upload-time = "2025-09-14T22:18:19.088Z" }, +]