Skip to content

feat: robot transport via zenoh#189

Merged
maxxgx merged 36 commits into
mainfrom
max/robot-transport
Jul 20, 2026
Merged

feat: robot transport via zenoh#189
maxxgx merged 36 commits into
mainfrom
max/robot-transport

Conversation

@maxxgx

@maxxgx maxxgx commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds Zenoh transport for sharing a physical robot between processes and over network.

flowchart LR
    subgraph Owner["Owner process"]
        HW["Robot driver"] <--> LOOP["Single write-first loop"]
    end
    subgraph Clients["Subscriber processes"]
        A["SharedRobot A"]
        B["SharedRobot B"]
    end

    LOOP -- "publish" --> STATE["/{name}/state"]
    ACTION["/{name}/action"] -- "latest action" --> LOOP
    LOOP -. "answer queries" .-> META["/{name}/metadata"]
    STATE --> A
    STATE --> B
    A --> ACTION
    A -. "probe/validate" .-> META
    B -. "probe/validate" .-> META
Loading
  • One owner exclusively connects to the robot.
  • SharedRobot instances create-or-attach by name.
  • Host-local name and device locks prevent ownership conflicts.
  • Local-only loopback is the default; remote mode is explicit.
  • State, actions, and metadata use low-latency, latest-wins Zenoh channels.

The design doc contains the full architecture. Please note that the design will be moved to the docs/design branch before this PR is merged (moved).

Benchmark Results

1. Teleop tick time: SharedRobot vs. direct driver

Single teleoperation runtime loop with one leader and one follower SO101:

Config n ticks mean p50 p99
Direct SO101 driver 14,244 2.53ms 2.52ms 2.77ms
SharedRobot 11,771 0.108ms 0.098ms 0.321ms

2. Pure Zenoh latency (no owner tick-loop)

Method n mean p50 p99 max
same-process pub/sub, 64B payload, busy-poll 1000 173.2us 43.7us 760.3us 1071.2us
IPC query round-trip vs. real owner subprocess 200 0.32ms 0.24ms 1.33ms 1.59ms

3. Cross-host Zenoh latency (RTT via query/reply)

Round-trip time measured on the requester's clock only (query out → reply back); one-way estimate is RTT/2. Host A connected via LAN and host B via WLAN on same router.

Method n RTT mean RTT p50 RTT p99 RTT max one-way estimate (mean/2)
Cross-host query/reply round trip 500 4.274ms 4.026ms 6.693ms 8.166ms 2.137ms
ping baseline (same path) 20 2.957ms 7.424ms 2.120 ms

Why

Reduce latency and allow sharing robot over same host or over the network.

Validation

Breaking changes

  • Introduces device_ids to the Robot protocol. All existing and future robot driver must implement it.

Related issues

Comment thread src/physicalai/robot/transport/_ids.py Outdated
Comment thread src/physicalai/robot/transport/_session.py Fixed
Comment thread src/physicalai/robot/transport/_owner_worker.py Fixed
@maxxgx
maxxgx marked this pull request as ready for review July 15, 2026 08:46
Copilot AI review requested due to automatic review settings July 15, 2026 08:46
@maxxgx
maxxgx requested review from a team as code owners July 15, 2026 08:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a Zenoh-based “shared robot” transport to Physical AI Runtime so a single owner process can exclusively control robot hardware while other processes attach to the same robot by name to read state and send actions. This extends the robot protocol with device_ids to support host-local locking and updates runtime telemetry encoding to a shared codec.

Changes:

  • Introduce physicalai.robot.transport (Zenoh owner/subscriber architecture, msgpack wire codec, deterministic session config, and host-local locking).
  • Extend Robot protocol with device_ids and implement it for SO101, WidowXAI, and test robots; add new robot transport error hierarchy.
  • Add extensive unit tests + documentation (how-to + design doc) and wire up CI/deps for the new transport extra.

Reviewed changes

Copilot reviewed 49 out of 50 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
uv.lock Adds eclipse-zenoh and msgpack locked deps; adjusts cffi marker.
pyproject.toml Adds transport extra deps (eclipse-zenoh, msgpack) and updates Ruff per-file ignores.
.github/workflows/library.yml Installs --extra transport in unit test workflow so new transport tests can run.
src/physicalai/robot/interface.py Extends the Robot protocol with device_ids.
src/physicalai/robot/errors.py Introduces structured robot error hierarchy for transport/connection/protocol issues.
src/physicalai/robot/init.py Re-exports new robot errors and lazily exposes SharedRobot.
src/physicalai/robot/so101/so101.py Implements device_ids for SO101 serial identity.
src/physicalai/robot/trossen/widowxai.py Implements device_ids for WidowXAI network identity.
src/physicalai/robot/trossen/bimanual_widowxai.py Implements composite device_ids for bimanual robot.
src/physicalai/robot/transport/init.py Public transport package entrypoint exporting SharedRobot + discovery.
src/physicalai/robot/transport/_ids.py Defines name validation, Zenoh key builders, and deterministic endpoint port derivation.
src/physicalai/robot/transport/_session.py Centralizes Zenoh session configuration (peer mode, local-only vs remote).
src/physicalai/robot/transport/_codec.py Implements msgpack + numpy wire codec for state/action/metadata.
src/physicalai/robot/transport/_importing.py Adds safe dotted-path importing helper for owner-side trusted config.
src/physicalai/robot/transport/_owner_config.py Adds serializable owner subprocess configuration and robot_class normalization.
src/physicalai/robot/transport/_lock.py Adds host-local name/device locking via flock-backed cache files and local registry.
src/physicalai/robot/transport/_owner.py Adds parent-side owner subprocess spawn + READY/ERROR handshake.
src/physicalai/robot/transport/_owner_worker.py Implements detached owner worker process and single-threaded control loop.
src/physicalai/robot/transport/_shared_robot.py Implements SharedRobot subscriber create-or-attach behavior, metadata validation, and action/state transport.
src/physicalai/runtime/observer/_codec.py Factors msgpack + numpy encoding/decoding into a reusable telemetry codec.
src/physicalai/runtime/observer/_telemetry.py Switches TelemetryEmitter packing to the shared observer codec and avoids runtime numpy import.
src/physicalai/runtime/observer/_subscriber.py Uses shared observer codec to decode numpy payloads and improves decode structure.
src/physicalai/inference/_importing.py Adds dotted-path import helper supporting nested attributes for inference components.
src/physicalai/inference/component_factory.py Uses shared import helper and validates imported objects are classes.
tests/unit/runtime/test_telemetry.py Updates telemetry numpy tests to use new observer codec and adjusts lint suppressions.
tests/unit/runtime/test_runtime.py Updates fake robot to satisfy new device_ids protocol requirement.
tests/unit/cli/test_cli.py Updates fake robot to satisfy new device_ids protocol requirement; adds config test for SharedRobot class_path usage.
tests/unit/inference/test_importing.py Adds tests for inference dotted-path importing (including nested attributes).
tests/unit/robot/test_protocol.py Extends Robot protocol tests to require device_ids.
tests/unit/robot/test_verify_robot.py Updates verify_robot test robots to satisfy new device_ids.
tests/unit/robot/test_so101.py Adds a test ensuring SO101 device ids are scheme-qualified and normalized.
tests/unit/robot/test_widowxai.py Adds a test ensuring WidowXAI device ids are scheme-qualified network addresses.
tests/unit/robot/test_bimanual_widowxai.py Adds a test ensuring bimanual robot device ids are unioned/deduplicated.
tests/unit/robot/transport/init.py Adds transport test package marker.
tests/unit/robot/transport/conftest.py Adds Zenoh availability gating + lock-dir isolation fixtures for transport tests.
tests/unit/robot/transport/fake.py Adds FakeRobot driver for transport tests with distinct shipped state semantics.
tests/unit/robot/transport/test_shared_robot.py Adds lifecycle, conflict, discovery, and protocol validation tests for SharedRobot.
tests/unit/robot/transport/test_session.py Adds unit tests asserting exact Zenoh config values for local-only vs allow_remote sessions.
tests/unit/robot/transport/test_owner_handshake.py Adds tests for owner startup phases, endpoint collision diagnostics, and error payloads.
tests/unit/robot/transport/test_owner_config.py Adds tests for owner config serialization and robot_class normalization/import rules.
tests/unit/robot/transport/test_lock.py Adds tests for host-local locks (name/device) including cross-process behavior.
tests/unit/robot/transport/test_latency.py Adds a slow test asserting p99 action-latency jitter stays within a budget.
tests/unit/robot/transport/test_importing.py Adds tests for transport dotted-path importing helper.
tests/unit/robot/transport/test_ids.py Adds tests for name validation, key building, endpoint port derivation, and host reporting.
tests/unit/robot/transport/test_codec.py Adds tests for transport msgpack codec (state/action/metadata) and numpy roundtrips.
docs/how-to/runtime/share-a-robot.md Adds user-facing how-to for SharedRobot usage, discovery, lifecycle, and security boundary.
docs/development/robot-zenoh-transport-design.md Adds design doc describing architecture, protocol, locking, QoS, and error model.
docs/development/security.md Documents transport trust boundary rules for unauthenticated remote actions (rule 12).
mkdocs.yml Adds the SharedRobot how-to page to the docs nav.
skills/runtime/physicalai-runtime-adding-a-robot-integration/references/robot-protocol.md Updates robot protocol reference docs to include device_ids.

Comment thread src/physicalai/robot/transport/_shared_robot.py
Comment thread src/physicalai/robot/transport/_shared_robot.py Outdated
Comment thread tests/unit/runtime/test_telemetry.py
@maxxgx maxxgx linked an issue Jul 15, 2026 that may be closed by this pull request
maxxgx and others added 4 commits July 15, 2026 11:06
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Comment thread docs/how-to/runtime/share-a-robot.md
Comment thread docs/how-to/runtime/share-a-robot.md
Comment thread docs/how-to/runtime/share-a-robot.md
Comment thread docs/how-to/runtime/share-a-robot.md
Comment thread docs/how-to/runtime/share-a-robot.md Outdated
Comment thread docs/development/robot-zenoh-transport-design.md Outdated
Comment thread docs/how-to/runtime/share-a-robot.md
Comment thread src/physicalai/robot/transport/_importing.py
Comment thread src/physicalai/robot/so101/so101.py Outdated
Comment thread src/physicalai/robot/transport/_client.py Outdated
Comment thread src/physicalai/robot/transport/_client.py
Comment thread src/physicalai/robot/transport/_client.py
Comment thread src/physicalai/robot/transport/_session.py
Comment thread src/physicalai/robot/interface.py
Comment thread src/physicalai/runtime/observer/_codec.py
MarkRedeman
MarkRedeman previously approved these changes Jul 17, 2026
Comment thread src/physicalai/robot/transport/_lock.py
Comment thread src/physicalai/robot/transport/_owner.py
Comment thread src/physicalai/robot/transport/_owner_config.py
Comment thread src/physicalai/robot/transport/_codec.py
Comment thread src/physicalai/robot/transport/_owner_worker.py
Comment thread src/physicalai/robot/transport/_shared_robot.py
Comment thread src/physicalai/robot/transport/_owner_worker.py
Comment thread docs/how-to/runtime/share-a-robot.md Outdated
@maxxgx
maxxgx merged commit d88ac1a into main Jul 20, 2026
22 checks passed
@maxxgx
maxxgx deleted the max/robot-transport branch July 20, 2026 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: shared robot transport (pub-sub)

5 participants