Multilinear Modelling of Faces and Expressions

In this work, we present a new versatile 3D multilinear statistical face model, based on a tensor factorisation of 3D face scans, that decomposes the shapes into person and expression subspaces. Investigation of the expression subspace reveals an inherent low-dimensional substructure, and further, a...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 2021-10, Vol.43 (10), p.3540-3554
Main Authors: Grasshof, Stella, Ackermann, Hanno, Brandt, Sami Sebastian, Ostermann, Jorn
Format: Article
Language:English
Subjects:
3D-reconstruction
Algorithms
Analytical models
Data models
expression transfer
HOSVD
Image reconstruction
Mathematical analysis
Modelling
Morphing
person transfer
Shape
Solid modeling
Statistical shape model
Subspaces
Tensile stress
tensor model
Tensors
Three-dimensional displays
Two dimensional displays
Two dimensional models
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this work, we present a new versatile 3D multilinear statistical face model, based on a tensor factorisation of 3D face scans, that decomposes the shapes into person and expression subspaces. Investigation of the expression subspace reveals an inherent low-dimensional substructure, and further, a star-shaped structure. This is due to two novel findings. (1) Increasing the strength of one emotion approximately forms a linear trajectory in the subspace. (2) All these trajectories intersect at a single point - not at the neutral expression as assumed by almost all prior works-but at an apathetic expression. We utilise these structural findings by reparameterising the expression subspace by the fourth-order moment tensor centred at the point of apathy. We propose a 3D face reconstruction method from single or multiple 2D projections by assuming an uncalibrated projective camera model. The non-linearity caused by the perspective projection can be neatly included into the model. The proposed algorithm separates person and expression subspaces convincingly, and enables flexible, natural modelling of expressions for a wide variety of human faces. Applying the method on independent faces showed that morphing between different persons and expressions can be performed without strong deformations.
ISSN:0162-8828
1939-3539
2160-9292
DOI:10.1109/TPAMI.2020.2986496