URML: validating a human-video-pretrained action against the target robot's envelope

[idoco2003]

Hi, disclosure first: this is AI-assisted prose I reviewed and approved before posting (background: https://github.com/URML-MARS/URML/blob/main/VIBE.md). Glad to switch to human-only if you prefer.

I maintain URML (https://urml.dev), a small Apache-2.0 language whose one job is to check an intended action against a robot's declared capabilities and a safety envelope before it runs.

VITRA is a natural fit because it learns manipulation from human-activity video, and what a human demonstrated is not automatically admissible on the specific robot that will execute it. URML can declare the target robot's reach, payload, gripper force, and the active keep-out and speed envelope, and validate the emitted action against that declaration before it drives the hardware. The check sits between the model output and the actuators, and touches neither the pretraining nor the policy.

URML does not learn, does not model, and does not replace VITRA. It is the static admissibility step that answers whether a given emitted action is inside the declared envelope for this robot.

Two questions:

1. For a VLA pretrained on human-activity video, is a declared-capability and envelope check on the emitted action a useful step before it drives a specific robot, or is embodiment feasibility already handled inside the model in practice?
2. Would a small worked example mapping a VITRA action onto a URML manifest, validated with no execution, be worth having?

Nothing here asks you to adopt, host, or maintain anything. Thanks for the work.

Ido Yahalomi (greenvh@gmail.com)

[arnoldland]

Hi, thanks for the discussion!

In our current setup, we expect the model to be fine-tuned with robot teleoperation data before deployment, so the policy output action is already adapted to the target robot's action space. In other words, the model is still a VLA that outputs robot-executable actions.

However, we agree that a VLA itself does not guarantee that every generated action can always be safely executed on a specific robot. There can still be constraints related to robot capability, workspace, collision, safety limits, etc., which may require additional verification.

From your description, it sounds like URML serves as a safety validation layer between the policy output and the robot execution. This is an interesting direction. However, we would like to better understand the distinction between URML and existing constraint-checking mechanisms already available in robotics frameworks, such as those in MoveIt, Pinocchio-based controllers, or ROS2 safety components.

Thanks again for sharing this idea!

[idoco2003]

Thanks, this is exactly the right question, and the honest answer is that URML does not replace those. It sits one layer up and is complementary.

MoveIt, Pinocchio, and the ROS 2 safety components are runtime mechanisms bound to ROS and to a specific robot model. MoveIt plans and collision-checks a trajectory against a live planning scene, Pinocchio computes the kinematics and dynamics a controller enforces, and the safety components monitor and stop at execution time. URML does not try to redo any of that.

The two differences are substrate-neutrality and staticness.

Substrate-neutral: URML is a specification, not a ROS library. The same capability manifest, envelope, and validator apply whether the target runs ROS 2, PX4, a vendor SDK, or bare firmware with no ROS at all. A policy that may deploy across several robots gets one declarative admissibility contract instead of a per-framework re-implementation. MoveIt cannot check a robot that does not run ROS.

Static and pre-dispatch: URML validates an intended action against a declared contract (reach, payload, gripper force, keep-out zones, speed envelope) before dispatch, with no planner or controller in the loop and fully offline. It answers "is this action admissible for this specific robot at all," cheaply, at the policy-output boundary, before the action reaches MoveIt or the controller for the geometric work.

So the division is: URML is the portable, declarative "is this within what this robot is declared able and allowed to do" gate; MoveIt, Pinocchio, and the ROS 2 safety stack are the runtime, ROS-coupled, geometric and dynamic mechanisms that do the fine-grained planning and monitoring. A real deployment uses both. URML does not do collision solving against a live scene and is not meant to.

If it is useful, I can put up a small worked example: one VITRA-style action checked against two different robot manifests, admissible on one and rejected on the other, with no ROS in the loop. That tends to make the boundary concrete faster than prose.

-- Ido

[idoco2003]

I went ahead and built the example I mentioned, in case it is useful: https://github.com/URML-MARS/URML/tree/main/examples/vitra

It validates one grasp action against two robot manifests that differ only in the gripper's force range. The same action is admissible on the higher-force arm and refused on the gentler one, decided statically, with no ROS. It is the concrete version of the distinction we were discussing.

No pressure to look. I just wanted the offer to be real rather than a promise.