Skip to content

fix: OpenVINO optimization notebook deployment flow in 003 notebook#184

Open
zhuo-yoyowz wants to merge 4 commits into
openvinotoolkit:docs/tutorialsfrom
zhuo-yoyowz:fix-003-openvino-optimization-notebook
Open

fix: OpenVINO optimization notebook deployment flow in 003 notebook#184
zhuo-yoyowz wants to merge 4 commits into
openvinotoolkit:docs/tutorialsfrom
zhuo-yoyowz:fix-003-openvino-optimization-notebook

Conversation

@zhuo-yoyowz

Copy link
Copy Markdown

Summary

Fixed issues in 003_OpenVINO_Optimization.ipynb, including:

  1. Fixed git clone Physical AI commit number, since the repo is developed fast and the last version used API out of date, causing errors during running.
  2. Removed the hard-coded robot calibration file path. Remind users to manually config the file path according to their env.
  3. Added a reminder for user to give permission to USB cameras ports in Linux OS.
  4. Added two choices for Model Loading, either using exported model already in local env or downloading an open-sourced pi-0.5 fine-tuned model.
  5. Updated some APIs according to the updated version of Physical API master branch.

@zhuo-yoyowz
zhuo-yoyowz requested a review from a team as a code owner July 7, 2026 04:54
@zhuo-yoyowz zhuo-yoyowz changed the title Fix OpenVINO optimization notebook deployment flow Fix: OpenVINO optimization notebook deployment flow in 003 notebook Jul 7, 2026

@maxxgx maxxgx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR.

Do you find these tutorials helpful? I was thinking we could reduce them, as they have quite a bit of overlap. For example, the collect_train_deploy notebook covers most of the topics.

Also, I believe it will be hard to keep this branch up to date, since breaking changes are quite regular while we're still in early development.

Comment on lines +52 to +54
"Clone the OpenVINO Physical AI repo and install a pinned revision.\n",
"\n",
"> **Why pin the revision?** The OpenVINO Physical AI APIs are evolving quickly. This notebook depends on a tested `physicalai` runtime revision, so the install cell below checks out a known-good commit before installing the package in editable mode. Update `PHYSICALAI_COMMIT` only after re-validating the rest of this notebook."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these tutorial were written before release. Now there is no need to clone the physicalai repo since it's already available on pypi, so you could pin a released version

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, just found the formal release version. Will modify the commit number to the released version.

@maxxgx maxxgx changed the title Fix: OpenVINO optimization notebook deployment flow in 003 notebook fix: OpenVINO optimization notebook deployment flow in 003 notebook Jul 10, 2026
@zhuo-yoyowz

Copy link
Copy Markdown
Author

Thanks for the PR.

Do you find these tutorials helpful? I was thinking we could reduce them, as they have quite a bit of overlap. For example, the collect_train_deploy notebook covers most of the topics.

Also, I believe it will be hard to keep this branch up to date, since breaking changes are quite regular while we're still in early development.

Thank you for your comments!
We do think Jupyter notebooks are helpful to developers, especially when they try to get hands-on to a whole new set of tools.
I tested all notebooks. Of course, 001 and 002 are documentations actually, therefore, in the future, maybe we could move them to Readme, so that developers could read them directly when navigating to the "tutorials" path?
004 notebook provides good pure simulation sample codes, which I think is also critical, especially to developers who currently don't own any physical robotic arms to test, but want to see how's everything working, whether it's hard to run and some initial performance results.
003 shows good optimization and deployment codes, but need some fixes, which I think could be also helpful when developers download an open-sourced model and would like to connect the robotic arms to run some initial inferences.
005collect_train_deploy does contain a full e2e process, however, it needs at least an A100 to run from the beginning, i.e. the model training. If only keeps this one notebook, developers could run nothing but have to leave the webpage soon if they don't own an A100.
So I hope these notebooks could be merged to the master branch, and we could provide clear guidance on how to use these notebooks to get started quickly and easily. I'm happy to be part of this for maintaining and upgrading, also writing tech blogs to promote these.

@maxxgx

maxxgx commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Thanks for the PR.
Do you find these tutorials helpful? I was thinking we could reduce them, as they have quite a bit of overlap. For example, the collect_train_deploy notebook covers most of the topics.
Also, I believe it will be hard to keep this branch up to date, since breaking changes are quite regular while we're still in early development.

Thank you for your comments! We do think Jupyter notebooks are helpful to developers, especially when they try to get hands-on to a whole new set of tools. I tested all notebooks. Of course, 001 and 002 are documentations actually, therefore, in the future, maybe we could move them to Readme, so that developers could read them directly when navigating to the "tutorials" path? 004 notebook provides good pure simulation sample codes, which I think is also critical, especially to developers who currently don't own any physical robotic arms to test, but want to see how's everything working, whether it's hard to run and some initial performance results. 003 shows good optimization and deployment codes, but need some fixes, which I think could be also helpful when developers download an open-sourced model and would like to connect the robotic arms to run some initial inferences. 005collect_train_deploy does contain a full e2e process, however, it needs at least an A100 to run from the beginning, i.e. the model training. If only keeps this one notebook, developers could run nothing but have to leave the webpage soon if they don't own an A100. So I hope these notebooks could be merged to the master branch, and we could provide clear guidance on how to use these notebooks to get started quickly and easily. I'm happy to be part of this for maintaining and upgrading, also writing tech blogs to promote these.

Thank you for your feedback. I think we still improvements before merging it to main branch:

@zhuo-yoyowz

Copy link
Copy Markdown
Author

Thanks for the PR.
Do you find these tutorials helpful? I was thinking we could reduce them, as they have quite a bit of overlap. For example, the collect_train_deploy notebook covers most of the topics.
Also, I believe it will be hard to keep this branch up to date, since breaking changes are quite regular while we're still in early development.

Thank you for your comments! We do think Jupyter notebooks are helpful to developers, especially when they try to get hands-on to a whole new set of tools. I tested all notebooks. Of course, 001 and 002 are documentations actually, therefore, in the future, maybe we could move them to Readme, so that developers could read them directly when navigating to the "tutorials" path? 004 notebook provides good pure simulation sample codes, which I think is also critical, especially to developers who currently don't own any physical robotic arms to test, but want to see how's everything working, whether it's hard to run and some initial performance results. 003 shows good optimization and deployment codes, but need some fixes, which I think could be also helpful when developers download an open-sourced model and would like to connect the robotic arms to run some initial inferences. 005collect_train_deploy does contain a full e2e process, however, it needs at least an A100 to run from the beginning, i.e. the model training. If only keeps this one notebook, developers could run nothing but have to leave the webpage soon if they don't own an A100. So I hope these notebooks could be merged to the master branch, and we could provide clear guidance on how to use these notebooks to get started quickly and easily. I'm happy to be part of this for maintaining and upgrading, also writing tech blogs to promote these.

Thank you for your feedback. I think we still improvements before merging it to main branch:

Thanks for the feedback.

I updated 003 to use the released PyPI package flow end-to-end instead of relying on a cloned local physicalai repo. The hardware extras are now installed from the pinned release as well.

I also replaced the personal Hugging Face repos with the official OpenVINO Physical AI collection model (OpenVINO/pi05-libero-fp16-ov) and added a guard to make clear that the official hub model is for model-loading validation unless the user verifies that its robot/camera/action schema matches their real SO101 setup.

For the broader 004 cleanup and browser-based simulation direction, I'll submit new PR to fix the issues. For the 002 notebook suggestion, yes, I think we could remove it and point users to documentation as you suggested. But for now, I think we need to show the tutorials repo first and could improve it step by step, so maybe this notebook could be removed later?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants