-
Notifications
You must be signed in to change notification settings - Fork 16
docs: rewrite README, update AGENTS.md, add 8 examples #87
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
f998b51
e2d6a0d
ca8db5a
7a70edb
5097531
316061d
b0e7088
807cf78
ca127ea
1f2d3f1
0a2e29c
dd6ae84
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| #!/usr/bin/env python3 | ||
| """Quickstart: 5 lines from import to simulation. | ||
|
|
||
| The Robot() factory auto-detects mode (defaults to sim when no hardware | ||
| is connected) and returns a ready-to-use MuJoCo simulation. | ||
|
|
||
| Requirements: | ||
| pip install strands-robots[sim-mujoco] | ||
|
|
||
| Usage: | ||
| python examples/01_sim_quickstart.py | ||
| """ | ||
|
|
||
| from strands_robots import Robot | ||
|
|
||
| # Create a simulated SO-100 arm — assets auto-download from MuJoCo Menagerie | ||
| sim = Robot("so100") | ||
|
|
||
| # Inspect the world | ||
| state = sim.get_state() | ||
| print(state["content"][0]["text"]) | ||
|
|
||
| # Step physics and render a frame | ||
| sim.step(n_steps=100) | ||
| frame = sim.render(width=640, height=480) | ||
| print(f"Rendered frame: {frame['content'][0]['text']}") | ||
|
|
||
| sim.destroy() | ||
| print("Done — simulation complete") |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| #!/usr/bin/env python3 | ||
| """The 5-Line Promise: natural language robot control. | ||
|
|
||
| Robot() returns an AgentTool with 64 simulation actions. Hand it to a | ||
| Strands Agent and control the robot through conversation. | ||
|
|
||
| Requirements: | ||
| pip install strands-agents strands-robots[sim-mujoco] | ||
|
|
||
| Usage: | ||
| python examples/02_sim_agent.py | ||
| """ | ||
|
|
||
| from strands import Agent | ||
|
|
||
| from strands_robots import Robot | ||
|
|
||
| # Robot("so100") auto-detects mode="sim", picks the "mujoco" backend, | ||
| # constructs a Simulation instance, calls add_robot() to load the SO-100 | ||
| # model (auto-downloading URDF/meshes on first run), and returns that | ||
| # Simulation as an AgentTool. You get full access to all Simulation | ||
| # actions — step(), render(), run_policy(), get_observation(), etc. | ||
| robot = Robot("so100") | ||
|
|
||
| # The sim IS the tool — pass it directly to Agent | ||
| agent = Agent(tools=[robot]) | ||
|
|
||
| # Natural language → simulation actions | ||
| result = agent("Get the simulation state, then run a mock policy for 1 second in fast mode") | ||
| print(result) | ||
|
|
||
| robot.destroy() | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| #!/usr/bin/env python3 | ||
| """Record simulation rollouts as LeRobot v3 datasets. | ||
|
|
||
| Runs a mock policy in MuJoCo, captures joint states + video, and saves | ||
| everything as a LeRobot-compatible dataset (parquet + AV1 video). | ||
|
|
||
| Note: | ||
| ``start_recording`` produces a LeRobotDataset (parquet + per-camera | ||
| MP4), so you need the ``lerobot`` extra in addition to ``sim-mujoco``. | ||
| For plain MP4 only (no dataset schema), use ``start_cameras_recording``. | ||
|
|
||
| Requirements: | ||
| pip install "strands-robots[sim-mujoco,lerobot]" | ||
|
|
||
| Usage: | ||
| python examples/03_sim_recording.py | ||
| """ | ||
|
|
||
| from strands_robots import Robot | ||
|
|
||
| sim = Robot("so100") | ||
|
|
||
| try: | ||
| # Start recording - creates LeRobot v3 dataset structure | ||
| sim.start_recording( | ||
| repo_id="local/so100_demo", | ||
| task="reach target", | ||
| fps=30, | ||
| root="/tmp/so100_dataset", | ||
| ) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
For the example to demonstrate what the docstring claims ("captures joint states + video, ... AV1 video"), add at least one camera before recording, e.g.: sim = Robot("so100")
sim.add_camera(name="topdown", position=[0, 0, 1.2], target=[0, 0, 0])
sim.start_recording(...)Otherwise reframe the docstring to say "joint states only" so users don't think they got an AV1 file. |
||
|
|
||
| # Run a mock policy (random actions) for 2 seconds. | ||
| # Video kwargs go inside the ``video`` dict, NOT as top-level args. | ||
| result = sim.run_policy( | ||
| robot_name="so100", | ||
| policy_provider="mock", | ||
| instruction="reach target", | ||
| duration=2.0, | ||
| fast_mode=True, | ||
| video={"path": "/tmp/so100_rollout.mp4", "fps": 30}, | ||
| ) | ||
| print(result["content"][0]["text"]) | ||
|
|
||
| # Finalize the episode | ||
| stop = sim.stop_recording() | ||
| print(stop["content"][0]["text"]) | ||
| finally: | ||
| sim.destroy() | ||
|
|
||
| print("Dataset saved to /tmp/so100_dataset/") | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| #!/usr/bin/env python3 | ||
| """Control real robot hardware with the same factory API. | ||
|
|
||
| Robot(mode="real") returns a HardwareRobot backed by LeRobot. The same | ||
| Agent workflow works — just swap the mode. | ||
|
|
||
| Requirements: | ||
| pip install strands-agents strands-robots[lerobot] | ||
| # Hardware: SO-100/SO-101 arm connected via USB (Feetech servos) | ||
|
|
||
| Usage: | ||
| # Auto-detect (switches to real if USB servo controller found) | ||
| STRANDS_ROBOT_MODE=real python examples/04_real_hardware.py | ||
|
|
||
| # Or set mode explicitly in code (see below) | ||
| """ | ||
|
|
||
| from strands import Agent | ||
|
|
||
| from strands_robots import Robot | ||
|
|
||
| # Explicit real mode with camera config | ||
| robot = Robot( | ||
| "so100", | ||
| mode="real", | ||
| cameras={ | ||
| "wrist": { | ||
| "type": "opencv", | ||
| "index_or_path": "/dev/video0", | ||
| "fps": 15, | ||
| "fourcc": "MJPG", | ||
| }, | ||
| }, | ||
| ) | ||
|
|
||
| # Same Agent interface as simulation | ||
| agent = Agent(tools=[robot]) | ||
| agent("Connect to the robot, read the current joint positions, and report status") | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No Compare to robot = None
try:
robot = Robot("so100", mode="real", cameras={...})
agent = Agent(tools=[robot])
agent("Connect to the robot, ...")
finally:
if robot is not None:
robot.cleanup() |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| #!/usr/bin/env python3 | ||
| """Run NVIDIA GR00T policy on real hardware. | ||
|
|
||
| Starts a GR00T inference server, connects to a real SO-101 arm with | ||
| dual cameras, and runs the policy through an Agent. | ||
|
|
||
| Requirements: | ||
| pip install strands-agents strands-robots[all] | ||
| # Hardware: SO-101 arm + 2 USB cameras | ||
| # Model: Download from HuggingFace (e.g., cagataydev/gr00t-wave) | ||
|
|
||
| Usage: | ||
| python examples/05_real_groot_policy.py | ||
| """ | ||
|
|
||
| from strands import Agent | ||
|
|
||
| from strands_robots import Robot, gr00t_inference, lerobot_camera, pose_tool | ||
|
|
||
| # Initialize robot to None so finally block is safe if construction fails | ||
| robot = None | ||
| groot_started = False | ||
|
|
||
| try: | ||
| # Real robot with dual cameras | ||
| robot = Robot( | ||
| "so101", | ||
| mode="real", | ||
| cameras={ | ||
| "wrist": { | ||
| "type": "opencv", | ||
| "index_or_path": "/dev/video0", | ||
| "fps": 15, | ||
| "fourcc": "MJPG", | ||
| }, | ||
| "front": { | ||
| "type": "opencv", | ||
| "index_or_path": "/dev/video2", | ||
| "fps": 15, | ||
| "fourcc": "MJPG", | ||
| }, | ||
| }, | ||
| ) | ||
|
|
||
| # Build agent with robot + inference tools | ||
| agent = Agent( | ||
| tools=[robot, gr00t_inference, lerobot_camera, pose_tool], | ||
| ) | ||
|
|
||
| # Start GR00T inference server | ||
| agent.tool.gr00t_inference( | ||
| action="start", | ||
| checkpoint_path="/data/checkpoints/gr00t-wave/checkpoint-300000", | ||
| port=5555, | ||
| data_config="so100_dualcam", | ||
| embodiment_tag="new_embodiment", | ||
| ) | ||
| groot_started = True | ||
|
|
||
| # Interactive control loop | ||
| print("GR00T policy running. Type instructions or 'quit' to exit.") | ||
| while True: | ||
| query = input("\n# ") | ||
| if query.lower() in ("quit", "exit", "q"): | ||
| break | ||
| agent(query) | ||
|
|
||
| finally: | ||
| # Always stop the inference server and release hardware, even on error. | ||
| if groot_started: | ||
| stop_result = gr00t_inference(action="stop", port=5555) | ||
| print(f"GR00T stop: {stop_result}") | ||
| if robot: | ||
| robot.cleanup() |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| #!/usr/bin/env python3 | ||
| """List all supported robots and their capabilities. | ||
|
|
||
| The registry contains 60+ robots with simulation assets and/or hardware | ||
| support. Use list_robots() to discover what's available. | ||
|
|
||
| Requirements: | ||
| pip install strands-robots | ||
|
|
||
| Usage: | ||
| python examples/06_list_robots.py | ||
| """ | ||
|
|
||
| from strands_robots import list_robots | ||
|
|
||
|
|
||
| def _flag(present: bool, label: str) -> str: | ||
| return f"[{label}]" if present else f"[{' ' * len(label)}]" | ||
|
|
||
|
|
||
| print("=== All Robots ===") | ||
| for r in list_robots(mode="all"): | ||
| sim_flag = _flag(r.get("has_sim", False), "sim") | ||
| real_flag = _flag(r.get("has_real", False), "real") | ||
| print(f" {sim_flag} {real_flag} {r['name']:25s} {r.get('description', '')}") | ||
|
|
||
| # Sim-capable: all robots that have simulation assets (includes those with | ||
| # hardware support too). For truly sim-only robots, filter out has_real. | ||
| sim_capable = list_robots(mode="sim") | ||
| sim_only = [r for r in sim_capable if not r.get("has_real", False)] | ||
| print(f"\n=== Sim-capable ({len(sim_capable)} robots, {len(sim_only)} sim-only) ===") | ||
| for r in sim_capable[:5]: | ||
| print(f" {r['name']}") | ||
|
|
||
| print(f"\n=== Real hardware ({len(list_robots(mode='real'))} robots) ===") | ||
| for r in list_robots(mode="real"): | ||
| print(f" {r['name']}") |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| #!/usr/bin/env python3 | ||
| """Run a HuggingFace ACT policy in MuJoCo and export a LeRobot dataset. | ||
|
|
||
| Downloads a pretrained ACT policy, runs it in simulation, records multi-camera | ||
| video + joint data as a LeRobot v3 dataset. | ||
|
|
||
| Requirements: | ||
| pip install "strands-robots[sim-mujoco,lerobot]" torch | ||
|
|
||
| Usage: | ||
| python examples/act_policy_simulation.py | ||
| """ | ||
|
|
||
| from strands_robots import Robot | ||
|
|
||
| # 1. Create simulated Aloha bimanual robot (14 actuators, 6 cameras) | ||
| sim = Robot("aloha") | ||
|
|
||
| # 2. Start recording a LeRobot dataset (parquet + AV1 video) | ||
| sim.start_recording( | ||
| repo_id="local/act_aloha_sim_demo", | ||
| task="transfer cube", | ||
| fps=50, | ||
| root="/tmp/act_aloha_dataset", | ||
| ) | ||
|
|
||
| # 3. Run a pretrained ACT policy from HuggingFace (~51M params). | ||
| # NOTE: This downloads model weights (~200MB) on first run. | ||
| # For a lightweight test, swap policy_provider="mock" and drop policy_config. | ||
| # Provider-specific kwargs (pretrained_name_or_path, device, ...) go inside | ||
| # ``policy_config``. Video output goes inside the ``video`` dict. | ||
| result = sim.run_policy( | ||
| robot_name="aloha", | ||
| policy_provider="lerobot_local", | ||
| policy_config={ | ||
| "pretrained_name_or_path": "lerobot/act_aloha_sim_transfer_cube_human", | ||
| }, | ||
| instruction="transfer cube", | ||
| duration=2.0, # seconds of sim time | ||
|
cagataycali marked this conversation as resolved.
|
||
| fast_mode=True, # no wall-clock sleep between steps | ||
| video={"path": "/tmp/act_aloha_rollout.mp4", "fps": 30}, | ||
| ) | ||
| print(result["content"][0]["text"]) | ||
|
|
||
| # 4. Save the episode to disk | ||
| stop = sim.stop_recording() | ||
| print(stop["content"][0]["text"]) | ||
|
|
||
| sim.destroy() | ||
Uh oh!
There was an error while loading. Please reload this page.