Skip to content

feat: add MediaPipe hand landmark support#2283

Open
tarunbommawar27 wants to merge 1 commit into
roboflow:developfrom
tarunbommawar27:feat/mediapipe-hand-skeleton
Open

feat: add MediaPipe hand landmark support#2283
tarunbommawar27 wants to merge 1 commit into
roboflow:developfrom
tarunbommawar27:feat/mediapipe-hand-skeleton

Conversation

@tarunbommawar27
Copy link
Copy Markdown
Contributor

Before submitting

Description

Adds support for MediaPipe hand landmarks in the key points module.

This PR adds a new Skeleton.HAND definition for MediaPipe's 21 hand landmarks and extends KeyPoints.from_mediapipe() to parse hand landmark outputs from both current and legacy MediaPipe result formats.

Type of Change

  • ✨ New feature (non-breaking change which adds functionality)
  • 🧪 Test update

Motivation and Context

MediaPipe hand landmarks contain 21 keypoints per hand. The existing Skeleton enum supported COCO, GHUM, and FaceMesh skeletons, but did not include a hand skeleton. This meant EdgeAnnotator could not automatically draw hand landmark connections.

KeyPoints.from_mediapipe() also handled pose and face landmarks, but did not handle hand landmarks. This PR adds support for both hand_landmarks and multi_hand_landmarks.

Closes #2170

Changes Made

  • Added Skeleton.HAND with 21 vertices and 20 MediaPipe hand landmark edges

  • Added automatic lookup support through existing SKELETONS_BY_VERTEX_COUNT and SKELETONS_BY_EDGE_COUNT

  • Extended KeyPoints.from_mediapipe() to support:

    • hand_landmarks
    • multi_hand_landmarks
  • Updated MediaPipe fake test helpers to support hand landmark test cases

  • Added tests for the hand skeleton definition and MediaPipe hand landmark parsing

  • Made MediaPipe confidence extraction robust by defaulting missing visibility values to 1.0

Testing

  • I have tested this code locally
  • I have added unit tests that prove my fix is effective or that my feature works
  • All new and existing targeted tests pass

Commands run:

python -m uv run pre-commit run --files src/supervision/key_points/core.py src/supervision/key_points/skeletons.py tests/helpers.py tests/key_points/test_core.py tests/key_points/test_skeletons.py

python -m uv run pytest tests/key_points/test_core.py::test_from_mediapipe_input tests/key_points/test_skeletons.py tests/key_points/test_annotators.py -q

Result:

35 passed

Google Colab (optional)

Not applicable.

Screenshots/Videos (optional)

Not applicable.

Additional Notes

The skeleton edges use 1-based indexing to match the existing EdgeAnnotator behavior, which subtracts 1 before indexing into keypoint arrays.

@tarunbommawar27 tarunbommawar27 requested a review from SkalskiP as a code owner May 29, 2026 21:11
@codecov
Copy link
Copy Markdown

codecov Bot commented May 29, 2026

Codecov Report

❌ Patch coverage is 75.00000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 79%. Comparing base (fb02e5c) to head (d154217).

Additional details and impacted files
@@           Coverage Diff           @@
##           develop   #2283   +/-   ##
=======================================
  Coverage       79%     79%           
=======================================
  Files           66      66           
  Lines         8567    8569    +2     
=======================================
+ Hits          6726    6732    +6     
+ Misses        1841    1837    -4     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Addition of hand landmarks as skeletons

1 participant