Human intention estimation based on hidden Markov model motion validation for safe flexible robotized warehouses

• Published in: Robotics and computer-integrated manufacturing 2019-06, Vol.57, p.182-196
• Main Authors: Petković, Tomislav, Puljiz, David, Marković, Ivan, Hein, Björn
• Format: Article
• Language: English
• Subjects: Algorithms; Augmented reality; Changing environments; Computer simulation; Hidden Markov model; Human behavior; Human intention estimation; Human motion; Logistics; Markov analysis; Markov chains; Path planning; Personnel management; Robots; Theory of Mind; Virtual reality; Voronoi graphs; Warehouses
• ISSN: 0736-5845; 1879-2537
• DOI: 10.1016/j.rcim.2018.11.004

•Human robot interaction is needed in modern warehouses.•Human intentions are estimated in industrial setup.•Experiments verified approach in augmented and virtual reality setups. With the substantial growth of logistics businesses the need for larger warehouses and their automation arises, thus using robots as assistants to human workers is becoming a priority. In order to operate efficiently and safely, robot assistants or the supervising system should recognize human intentions in real-time. Theory of Mind (ToM) is an intuitive human conception of other humans’ mental state, i.e., beliefs and desires, and how they cause behavior. In this paper we propose a ToM based human intention estimation algorithm for flexible robotized warehouses. We observe human’s, i.e., worker’s motion and validate it with respect to the goal locations using generalized Voronoi diagram based path planning. These observations are then processed by the proposed hidden Markov model framework which estimates worker intentions in an online manner, capable of handling changing environments. To test the proposed intention estimation we ran experiments in a real-world laboratory warehouse with a worker wearing Microsoft Hololens augmented reality glasses. Furthermore, in order to demonstrate the scalability of the approach to larger warehouses, we propose to use virtual reality digital warehouse twins in order to realistically simulate worker behavior. We conducted intention estimation experiments in the larger warehouse digital twin with up to 24 running robots. We demonstrate that the proposed framework estimates warehouse worker intentions precisely and in the end we discuss the experimental results.