Real-Time Interaction for 3D Pixel Human in Virtual Environment

Conducting realistic interactions while communicating efficiently in online conferences is highly desired but challenging. In this work, we propose a novel pixel-style virtual avatar for interacting with virtual objects in virtual conferences that can be generated in real-time. It consists of a 2D s...

Full description

Saved in:
Bibliographic Details
Published in:Applied sciences 2023-01, Vol.13 (2), p.966
Main Authors: Deng, Haoke, Zhang, Qimeng, Jin, Hongyu, Kim, Chang-Hun
Format: Article
Language:English
Subjects:
Avatars
Cameras
Communication
Conferences
Meetings
Methods
pixel-style art
Pixels
Pose estimation
Real time
Realism
Semantics
Sensors
Video data
Video teleconferencing
virtual avatar
Virtual environments
virtual meeting
Virtual reality
Webcams
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Conducting realistic interactions while communicating efficiently in online conferences is highly desired but challenging. In this work, we propose a novel pixel-style virtual avatar for interacting with virtual objects in virtual conferences that can be generated in real-time. It consists of a 2D segmented head video stream for real-time facial expressions and a 3D point cloud body for realistic interactions, both of which are generated from RGB video input of a monocular webcam. We obtain a human-only video stream with a human matting method and generate the 3D avatar’s arms with a 3D pose estimation method, which improves the stereoscopic realism and sense of interaction of conference participants while interacting with virtual objects. Our approach fills the gap between 2D video conferences and 3D virtual avatars and combines the advantages of both. We evaluated our pixel-style avatar by conducting a user study; the result proved that the efficiency of our method is superior to other various existing avatar types.
ISSN:2076-3417
2076-3417
DOI:10.3390/app13020966