Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Repository files navigation

AutonomusDroneHandGesture

This is a sample program which allows users to control an Airsim drone using hand gestures.

This repository contains the following contents:

  • Sample program
  • Airsim gesture control interface
  • Hand sign recognition model(TFLite)
  • Finger gesture recognition model(TFLite)
  • Learning data for hand sign recognition and notebook for learning
  • Learning data for finger gesture recognition and notebook for learning

Demo

Here's how to run the sample program

gesture_contoller.py

When your webcam is turned on, you can use gesture to control the Airsim Drone

Directory

│  gesture_controller.py
│  airsim_controller.py
│  KeyController.py
│  keypoint_classification.ipynb
│  keypoint_classification_EN.ipynb
│  point_history_classification.ipynb
│  requirements.txt
│  app.py
│
├─model
│  ├─keypoint_classifier
│  │  │  keypoint.csv
│  │  │  keypoint_classifier.hdf5
│  │  │  keypoint_classifier.py
│  │  │  keypoint_classifier.tflite
│  │  └─ keypoint_classifier_label.csv
│  │          
│  └─point_history_classifier
│      │  point_history.csv
│      │  point_history_classifier.hdf5
│      │  point_history_classifier.py
│      │  point_history_classifier.tflite
│      └─ point_history_classifier_label.csv
│          
├─utils
│   └─cvfpscalc.py
│
└─airsim_functions
    └─orbit.py


Gestures

For the keypoint classifier

0 Stop

image

1 Front

front

2 Back

back

3 Up

image

4 Down

image

5 Left

image

6 Right

image

7 Pointer (For turning clockwise and anticlockwise)

image

For the point history classifier

0 Turning clockwise

ezgif com-video-to-gif

1 Turning anticlockwise

ezgif com-video-to-gif (1)

Airsim

Documentation

https://microsoft.github.io/AirSim/

Assets

https://github.com/microsoft/AirSim/releases

Model training

To record new keypoint gesture run app.py

  • Press K key to enter gesture recording mode
  • Press number key (0 ~ 9) to record the coordinates of hand.
  • In the keypoint.csv the first column is the label, which is the number you pressed.
  • Subsequent column is the coordinates, an array flattened to single row, of keypoints.

To record new point history gesture run app.py

  • Press H key to enter gesture recording mode
  • Press number key (0 ~ 9) to record the history of your point right now and 15 frames before.
  • In the point_history.csv the first column is the label, which is the number you pressed.
  • Subsequent column is the coordinates, an array flattened to single row, of the history of the point that you are tracking.

More info could be found on https://github.com/kinivi/hand-gesture-recognition-mediapipe

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages