Skip to content

feat: add experimental QCS8550 QNN deployment for LIBERO Object - #9

Open
sunwen-me wants to merge 1 commit into
H-EmbodVis:mainfrom
sunwen-me:feat/qcs8550-qnn-deployment-20260817
Open

feat: add experimental QCS8550 QNN deployment for LIBERO Object#9
sunwen-me wants to merge 1 commit into
H-EmbodVis:mainfrom
sunwen-me:feat/qcs8550-qnn-deployment-20260817

Conversation

@sunwen-me

Copy link
Copy Markdown

Summary

Adds an experimental, reproducible Qualcomm QCS8550 HTP v73 / QAIRT 2.48 deployment path for the released LIBERO Object checkpoint.

Features

  • Exports fixed-shape DINOv3, BERT, and policy-core ONNX graphs and submits them to Qualcomm AI Hub for QNN context-binary compilation.
  • Runs the five contexts through a persistent board-native QNN service, avoiding per-request context creation and process launch.
  • Provides a host-side LIBERO adapter, native replay client, rollout entry point, and an AI-agent deployment runbook with explicit safety and validation stop conditions.
  • Preserves issue Unable to reproduce reported LIBERO-Spatial/Object success rates with released checkpoints #4 behavior by exporting DINO outputs.hidden_states[-1] with transformers==4.56.*.

Files

Path Purpose
deployment/qcs8550/ QNN export, compilation, deployment, replay, and LIBERO rollout support.
deployment/qcs8550/AGENT_DEPLOYMENT.md End-to-end AI-agent deployment runbook, including AI Hub credential handling and failure triage.
README.md Links the experimental QCS8550 deployment documentation.

Model weights, ONNX exports, QNN context binaries, AI Hub job manifests, replay artifacts, and credentials are intentionally excluded.

Testing

  • Python syntax validation for the deployment tools and host adapter.
  • Native QNN service compiled against QAIRT 2.48 headers; usage path verified.
  • Board bring-up on QCS8550 / QAIRT 2.48: persistent service matched the same compiled-context board replay over 20 warm requests (service P50/P95: 60.1/67.6 ms).
  • One LIBERO Object episode for each task 0-9 succeeded. This is a bring-up smoke test, not the upstream 50-trial-per-task evaluation.

Numerical Caveat

HTP output is not asserted to be FP32-exact. On the contributor setup, real-sample DINO relative L2 drift was 18.65% / 9.46% for the two views, BERT was 4.73%, and final normalized action drift was 1.10%. The implementation validates board replay stability and rollout behavior rather than claiming framework-level numerical equivalence.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant