Co-Recognition of Multiple Fingertips for Tabletop Human-Projector Interaction

• Published in: IEEE transactions on multimedia 2019-06, Vol.21 (6), p.1487-1498
• Main Authors: Choi, Ouk, Son, Young-Jun, Lim, Hwasup, Ahn, Sang Chul
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Cameras; depth camera; fingertip; Forestry; Human-projector interaction; Indexes; Pose estimation; Projectors; random forest; Recognition; Surface emitting lasers; Three-dimensional displays; Thumb
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2018.2880608

We present a depth-based fingertip recognition method for interactive projectors. We use a depth camera attached to a projector, so it is possible to change the relative pose between the projector and the projection surface without manual recalibration. For detection and classification of fingertips, we propose using cascaded random forests boosted by our 3-D pose-normalized pixel-difference features. The ensemble probabilities from the cascaded random forests are used to define a score function of a subset of detected fingertips. By finding the subset maximizing the score function, the fingertips in the subset are correctly classified, and the remaining incorrectly detected fingertips are rejected. Experiments show that the proposed method outperforms conventional random forest and convolutional neural network classifiers. In addition, our developed applications show the advantage of the proposed method in assigning different roles to different fingers.