Attribute And-Or Grammar for Joint Parsing of Human Pose, Parts and Attributes

• Published in: IEEE transactions on pattern analysis and machine intelligence 2018-07, Vol.40 (7), p.1555-1569
• Main Authors: Seyoung Park, Nie, Bruce Xiaohan, Song-Chun Zhu
• Format: Article
• Language: English
• Subjects: Adult; Aged; And-Or grammar; Attribute grammar; attribute prediction; Body parts; Child; Databases, Factual; Decomposition; Dependence; Female; Formulations; Geometry; Glass; Grammar; Graphical representations; Head; Human body; Humans; Image Processing, Computer-Assisted - methods; joint parsing; Magnetic heads; Male; Mathematical models; Modelling; Models, Theoretical; part localization; Pattern Recognition, Automated - methods; Pose estimation; Posture - physiology; Predictive models; Structural hierarchy; Terminology as Topic
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2017.2731842

This paper presents an attribute and-or grammar (A-AOG) model for jointly inferring human body pose and human attributes in a parse graph with attributes augmented to nodes in the hierarchical representation. In contrast to other popular methods in the current literature that train separate classifiers for poses and individual attributes, our method explicitly represents the decomposition and articulation of body parts, and account for the correlations between poses and attributes. The A-AOG model is an amalgamation of three traditional grammar formulations: (i) Phrase structure grammar representing the hierarchical decomposition of the human body from whole to parts; (ii) Dependency grammar modeling the geometric articulation by a kinematic graph of the body pose; and (iii) Attribute grammar accounting for the compatibility relations between different parts in the hierarchy so that their appearances follow a consistent style. The parse graph outputs human detection, pose estimation, and attribute prediction simultaneously, which are intuitive and interpretable. We conduct experiments on two tasks on two datasets, and experimental results demonstrate the advantage of joint modeling in comparison with computing poses and attributes independently. Furthermore, our model obtains better performance over existing methods for both pose estimation and attribute prediction tasks.