Deployment pipeline for SmolVLA on a real SO-101 arm (Qualcomm Dragonwing IQ-9075, ARM CPU) #31

Activity

1. khanhnd61-vr commented on Sep 30, 2026

   

   khanhnd61-vr

   on Sep 30, 2026

   Collaborator

   More actions

   Thanks for the detailed write-up - you've identified the gap accurately.
   No real-robot bridge in this repo yet - eval/client/ only targets LIBERO and SimplerEnv.

   We're building one in a LeRobot fork

   • A CLI wiring LeRobot's robot interface to the vla-server protocol.
   • Covers both vla.cpp and our latest project vla.simd engine (engine to inference VLA on CPU).
   • SmolVLA on an SO-101 is the target case.

   Two caveats:

   • Experimental and not mature yet.
   • Async inference is not supported yet. vla-server is synchronous REQ/REP, so the control loop is synchronous too: a chunk is executed before the next request goes out, with no overlap of inference and execution.

   We'll comment here once it's ready- should be soon.
   Benchmark numbers from the Dragonwing IQ-9075 would be very welcome.

   👍React with 👍1Reacted by George Igwegbe
2. ravediamond commented on Oct 5, 2026

   

   ravediamond

   on Oct 5, 2026

   Contributor

   More actions

   Here are some real-arm numbers for the new client, in case they're useful.

   Setup: Jetson Orin NX 16 GB (JetPack 6.2, MAXN_SUPER), vla.cpp f7e0f7f built on the board, and my own SmolVLA fine-tune on an SO-101 (front + wrist cameras), converted with convert_smolvla_to_gguf.py. I plugged VlaCppClient into my LeRobot rollout harness, so PyTorch and vla.cpp share the same robot I/O, recording and labelling. Task "Put the blue bowl in the pink bowl.", 25 s trials at 30 Hz, 50-action chunks, 10 trials each.

   	PyTorch	vla.cpp	vla.cpp --flash-attn --mm-prec default
   Success	9/10	9/10	8/10
   Chunk inference	974 ms	390 ms	238 ms
   Loop blocked on inference	39%	20%	14%
   Control rate	19.3 Hz	24.6 Hz	26.6 Hz

   All failures were missed grasps, and at 10 trials the success rates are within noise.

   A few notes:

   • Replay mode was a nice sanity check before letting the arm move.
   • The loop still waits ~20% of each trial, which is what Async serving: ROUTER socket, predict thread and latest-wins request queue #34 should fix. I can run the same 10 trials with Async serving: ROUTER socket, predict thread and latest-wins request queue #34 and the async client on the Orin NX if that helps.
   • On JetPack 6 (Ubuntu 22.04) there's no cppzmq-dev package, so the README's apt line fails. Putting cppzmq's two headers in a folder and passing -DCPPZMQ_INCLUDE_DIR works. Happy to send a small docs PR.
   • I also ran the full vla-bench sweep on the Orin NX and will open a PR with the report.