MediaPipe Hands: On-device Real-time Hand Tracking

Andrey Vakunov
Chuo-Ling Chang
Fan Zhang
Valentin Bazarevsky
CV4ARVR 2020 (2020)
Google Scholar

Abstract

We present a real-time on-device hand tracking pipeline that predicts hand skeleton from only single camera input for AR/VR applications. The pipeline consists of two models: 1) a palm detector, 2) a hand landmark prediction. It's implemented via MediaPipe which is a cross-platform ML pipeline. The proposed architecture demonstrates realtime inference speed on mobile GPUs and a high prediction quality. MediaPipe Hands is open-sourced at https://github.com/google/mediapipe.