This is a sample Unity (2019.4.10f1) Plugin to use Mediapipe.
Platforms
Unity support would be an excellent approach. Especially for variety of Augmented Reality apps and solutions, the Mediapipe Plugin for Unity would be a great help. TheBricktop commented on Feb 9, 2020 I kinda find this weird that feature request is closed without any answear. Initializing unity; mediapipe.
Mediapipe-unity-hand-tracking; Starrers; 9 starrers: 9 public and 0 private Name. Sort by Name Name, descending Recently starred Oldest starred Arda Satata Fitriajie. Mediapipe-unity-hand-tracking Project overview Project overview Details Activity Releases Repository Repository Files Commits Branches Tags Contributors Graph.
Prerequisites
MediaPipe
Please be sure to install required packages and check if you can run the official demos on your machine.
OpenCV
By default, it is assumed that you use OpenCV 3 and it is installed under /usr (e.g. /usr/lib/libopencv_core.so).If your version or path is different, please edit C/third_party/opencv_linux.BUILD and C/WORKSPACE.
Protocol Buffer
The protocol buffer compiler is required.It is also necessary to install .NET Core SDK(3.x) and .NET Core runtime 2.1 to build Google.Protobuf.dll.
Build
You may want to edit BUILD file before building so as to only include necessary calculators to reduce the library size.For more information, please see the BUILD file.
Models
The models used in example scenes are copied under Assets/Mediapipe/SDK/Models by running make install.
If you’d like to use other models, you should place them so that Unity can read.For example, if your graph depends on face_detection_front.tflite, then you can place the model file under Assets/Mediapipe/SDK/Models/ and set the path to the model_path value in your config file.
If neccessary, you can also change the model paths for subgraphs (e.g. FaceDetectionFrontCpu) by updating mediapipe_model_path.diff.
Example Scenes
- Hello World!
- Face Detection (on CPU/GPU)
- Face Mesh (on CPU/GPU)
- Iris Tracking (on CPU/GPU)
- Hand Tracking (on CPU/GPU)
- Pose Tracking (on CPU/GPU)
- Hair Segmentation (on GPU)
- Object Detection (on CPU/GPU)
Troubleshooting
DllNotFoundException: mediapipe_c
OpenCV’s path may not be configured properly.
If you’re sure the path is correct, please check on Load on startup in the plugin inspector, click Apply button, and restart Unity Editor.Some helpful logs will be output in the console.
InternalException: INTERNAL: ; eglMakeCurrent() returned error 0x3000
If you encounter an error like below and you use OpenGL Core as the Unity’s graphics APIs, please try Vulkan.
TODO
LICENSE
MIT
MediaPipe offers open source cross-platform, customizable ML solutions for live and streaming media.
Google Mediapipe Unity
Solutions
Explore what is possible with MediaPipe today
Human Pose Detection and Tracking
High-fidelity human body pose tracking, inferring up to 33 3D full-body landmarks from RGB video frames
Face Mesh
468 face landmarks in 3D with multi-face support
Hand Tracking
21 landmarks in 3D with multi-hand support, based on high-performance palm detection and hand landmark model
Mediapipe Unity Tutorial
Holistic Tracking
Simultaneous and semantically consistent tracking of 33 pose, 21 per-hand, and 468 facial landmarks
Hair Segmentation
Super realistic real-time hair recoloring
Object Detection and Tracking
Detection and tracking of objects in video in a single pipeline
Mediapipe Unity Player
Face Detection
Ultra lightweight face detector with 6 landmarks and multi-face support
Iris Tracking and Depth Estimation
Accurate human iris tracking and metric depth estimation without a specialized hardware. Tracks iris, pupil and the eye contour landmarks.
3D Object Detection
Detection and 3D pose estimation of everyday objects like shoes and chairs
And More Solutions
See code samples on how to run MediaPipe on mobile (Android/iOS), desktop/server and Edge TPU
End-to-end acceleration
Built-in fast ML inference and processing accelerated even on common hardware
Build once, deploy anywhere
Unified solution works across Android, iOS, desktop/cloud, web and IoT
Free and open source
Framework and solutions both under Apache 2.0, fully extensible and customizable
Used in leading ML products and teams
“MediaPipe has supercharged our work on vision and hearing features for Nest Hub Max, allowing us to bring features like Quick Gestures to our users.”
Geremy Heitz
Google Nest
“The reusability of MediaPipe components and how easy it is to swap out inputs/outputs saved us a lot of time on preparing demos for different customers.”
Henry Tran
Google Cloud
“MediaPipe has made it extremely easy to build our 3D person pose reconstruction demo app, facilitating accelerated neural network inference on device and synchronization of our result visualization with the video capture stream. Highly recommended!”
George Papandreou
Ariel AI, CTO
“MediaPipe is one of the most widely shared and re-usable libraries for media processing within Google.”
Kahye Song
Mediapipe Unity Free
Google Nest Cam