On-device Real-time Hand Gesture Recognition

Chuo-Ling Chang
Esha Uboweja
George Sung
Kanstantsin Sokal
Valentin Bazarevsky
ICCV Workshop on Computer Vision for Augmented and Virtual Reality, Montreal, Canada, 2021 (2021)
Google Scholar

Abstract

We present an on-device real-time hand gesture recogni-tion (HGR) system, which detects a set of predefined staticgestures from a single RGB camera. The system consists oftwo parts: a hand skeleton tracker and a gesture classifier.We improve and extend MediaPipe Hands [12] for the handtracker. We experiment with two different gesture classifiers,one heuristics based and one neural network (NN) based.