Human Action Performance Using Deep Neuro-Fuzzy Recurrent Attention Model

• Published in: IEEE access 2020, Vol.8, p.57749-57761
• Main Authors: Bendre, Nihar, Ebadi, Nima, Prevost, John J., Najafirad, Peyman
• Format: Article
• Language: English
• Subjects: Adaptation models; artificial intelligence; Artificial neural networks; Attention mechanism; behavior analysis; Computer vision; convolutional neural networks; Deep learning; Fuzzy logic; human action recognition; Human activity recognition; Human motion; Human performance; Indexing; intensity indexing; Kinetic theory; machine learning; neurofuzzy systems; recurrent neural networks; supervised learning; Uncertainty
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2020.2982364

A great number of computer vision publications have focused on distinguishing between human action recognition and classification rather than the intensity of actions performed. Indexing the intensity which determines the performance of human actions is a challenging task due to the uncertainty and information deficiency that exists in the video inputs. To remedy this uncertainty, in this paper we coupled fuzzy logic rules with the neural-based action recognition model to rate the intensity of a human action as intense or mild. In our approach, we used a Spatio-Temporal LSTM to generate the weights of the fuzzy-logic model, and then demonstrate through experiments that indexing of the action intensity is possible. We analyzed the integrated model by applying it to videos of human actions with different action intensities and were able to achieve an accuracy of 89.16% on our intensity indexing generated dataset. The integrated model demonstrates the ability of a neuro-fuzzy inference module to effectively estimate the intensity index of human actions.