Assessment of the handcart pushing and pulling safety by using deep learning 3D pose estimation and IoT force sensors

• Published in: Expert systems with applications 2021-11, Vol.183, p.115371, Article 115371
• Main Authors: Vukicevic, Arso M., Macuzic, Ivan, Mijailovic, Nikola, Peulic, Aleksandar, Radovic, Milos
• Format: Article
• Language: English
• Subjects: Algorithms; Deep learning; Electroencephalography; Ergonomics; Handcart; Machine learning; Motion sensors; Occupational safety; Pose estimation; Pushing; Pushing and pulling; Safety management; Sensors; Torso; Unsafe acts
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2021.115371

[Display omitted] •Manual safety management of pushing and pulling (P&P) tasks is inefficient.•IoT force sensors were used to assess P&P forces.•Safety of P&P acts was assessed from 3D poses obtained with the VIBE algorithm.•Besides increased forces, unsafe P&P acts are correlated with the P&P momentum.•Future studies should account for turn-points and loading/unloading of cargo. Pushing and pulling (P&P) are common and repetitive tasks in industry, which non-ergonomic execution is among major causes of musculoskeletal disorders (MSD). The current safety management of P&P assumes restrictions of maximal weight, distance, height – while variable individual parameters (such as the P&P pose ergonomic) remain difficult to account for with the standardized guides. Since manual detection of unsafe P&P acts is subjective and inefficient, the aim of this study was to utilize IoT force sensors and IP cameras to detect unsafe P&P acts timely and objectively. Briefly, after the IoT module detects moments with increased P&P forces, the assessment of pose ergonomics was performed from the employee pose reconstructed with the VIBE algorithm. The experiments showed that turn-points correspond to the high torsion of torso, and that in such moments poses are commonly non ergonomic (although P&P forces are below values defined as critical in previous studies – their momentum cause serious load on the human body). Moreover, the analysis revealed that the loading/unloading of a cargo are also moments of frequent unsafe P&P acts – although they are commonly neglected when studying P&P. The experimental validation of the solution showed good agreement with motion sensors and high potential for monitoring and improving P&P workplace safety. Accordingly, future research will be directed towards: 1) acquisition of P&P data sets for direct recognition and classification of unsafe P&P acts; 2) incorporation of wearable sensors (EMG and EEG) for detecting fatigue and decrease of physical abilities.