A hierarchical 3D-motion learning framework for animal spontaneous behavior mapping

• Published in: Nature communications 2021-05, Vol.12 (1), p.2784-2784, Article 2784
• Main Authors: Huang, Kang, Han, Yaning, Chen, Ke, Pan, Hongli, Zhao, Gaoyang, Yi, Wenling, Li, Xiaoxi, Liu, Siyuan, Wei, Pengfei, Wang, Liping
• Format: Article
• Language: English
• Subjects: 631/1647/2198; 631/378/116; 631/601/18; 64/60; Animal behavior; Animal diseases; Animal health; Animals; Behavior, Animal - physiology; Biomechanical Phenomena - physiology; Dynamics; Humanities and Social Sciences; Kinematics; Learning algorithms; Machine Learning; Mice; Mice, Inbred C57BL; Mice, Knockout; Microfilament Proteins - genetics; Motion capture; Movement - physiology; multidisciplinary; Multilayers; Nerve Tissue Proteins - genetics; Neural networks; Phenotypes; Phenotyping; Science; Science (multidisciplinary); Structural hierarchy; Three dimensional motion; Transgenic animals; Video Recording - methods
• ISSN: 2041-1723; 2041-1723
• DOI: 10.1038/s41467-021-22970-y

Animal behavior usually has a hierarchical structure and dynamics. Therefore, to understand how the neural system coordinates with behaviors, neuroscientists need a quantitative description of the hierarchical dynamics of different behaviors. However, the recent end-to-end machine-learning-based methods for behavior analysis mostly focus on recognizing behavioral identities on a static timescale or based on limited observations. These approaches usually lose rich dynamic information on cross-scale behaviors. Here, inspired by the natural structure of animal behaviors, we address this challenge by proposing a parallel and multi-layered framework to learn the hierarchical dynamics and generate an objective metric to map the behavior into the feature space. In addition, we characterize the animal 3D kinematics with our low-cost and efficient multi-view 3D animal motion-capture system. Finally, we demonstrate that this framework can monitor spontaneous behavior and automatically identify the behavioral phenotypes of the transgenic animal disease model. The extensive experiment results suggest that our framework has a wide range of applications, including animal disease model phenotyping and the relationships modeling between the neural circuits and behavior. Animal behavior usually has a hierarchical structure and dynamics. Here, the authors propose a parallel and multi-layered framework to learn the hierarchical dynamics and generate an objective metric to map the behaviour into the feature space.