Light direction estimation and hand touchable interaction for augmented reality

Augmented reality is a technology that combines a virtual world with the real world. How to improve the realism of augmented reality is an important topic. One focus of this paper is lighting consistency between virtual and real world, and the other is interaction with virtual object using hands. Es...

Full description

Saved in:
Bibliographic Details
Published in:Virtual reality : the journal of the Virtual Reality Society 2022-09, Vol.26 (3), p.1155-1172
Main Authors: Liu, Damon Shing-Min, Wu, Shao-Jun
Format: Article
Language:English
Subjects:
Artificial Intelligence
Augmented reality
Computer Graphics
Computer Science
Elevation angle
Estimation
Gesture recognition
Image Processing and Computer Vision
Light
Lighting
Object recognition
Original Article
Realism
Shadows
Touch screens
User Interfaces and Human Computer Interaction
Virtual reality
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Augmented reality is a technology that combines a virtual world with the real world. How to improve the realism of augmented reality is an important topic. One focus of this paper is lighting consistency between virtual and real world, and the other is interaction with virtual object using hands. Estimating lighting conditions through traditional methods often requires many prior knowledge of the scene. We propose a method that estimates the light direction based on shadows and foreground objects with only one scene image. We detect and calculate the relative direction of an object and its shadow in the scene to estimate the azimuth of the light, and use area size ratio of the object and its shadow to estimate the elevation angle of the light. We used some real scenes to test our method. However, the exact light direction of the real world is difficult to acquire, so we further verified our method by establishing a number of virtual scenes with preset light direction. Moreover, hand gesture-based human–computer interaction provides a natural and easy way for interaction. Traditional augmented reality interactions use markers or touch screens. We apply gesture recognition and hand touchable interaction in augmented reality, allowing the user’s hand to occlude the virtual object in the picture, and recognize the gesture. By adding estimated light and hand touchable interactions, we enhance the realism of augmented reality.
ISSN:1359-4338
1434-9957
DOI:10.1007/s10055-022-00624-8