Visual accuracy dominates over haptic speed for state estimation of a partner during collaborative sensorimotor interactions

• Published in: Journal of neurophysiology 2023-07, Vol.130 (1), p.23-42
• Main Authors: Lokesh, Rakshith, Sullivan, Seth R, St Germain, Laura, Roth, Adam M, Calalo, Jan A, Buggeln, John, Ngo, Truc, Marchhart, Vanessa R F, Carter, Michael J, Cashaback, Joshua G A
• Format: Article
• Language: English
• Subjects: Computer Simulation; Feedback, Sensory; Hand; Haptic Technology; Humans; Movement
• ISSN: 0022-3077; 1522-1598
• DOI: 10.1152/jn.00053.2023

We routinely have physical interactions with others, whether it be handing someone a glass of water or jointly moving a heavy object together. These sensorimotor interactions between humans typically rely on visual feedback and haptic feedback. Recent single-participant studies have highlighted that the unique noise and time delays of each sense must be considered to estimate the state, such as the position and velocity, of one's own movement. However, we know little about how visual feedback and haptic feedback are used to estimate the state of another person. Here, we tested how humans utilize visual feedback and haptic feedback to estimate the state of their partner during a collaborative sensorimotor task. Across two experiments, we show that visual feedback dominated haptic feedback during collaboration. Specifically, we found that visual feedback led to comparatively lower task-relevant movement variability, smoother collaborative movements, and faster trial completion times. We also developed an optimal feedback controller that considered the noise and time delays of both visual feedback and haptic feedback to estimate the state of a partner. This model was able to capture both lower task-relevant movement variability and smoother collaborative movements. Taken together, our empirical and modeling results support the idea that visual accuracy is more important than haptic speed to perform state estimation of a partner during collaboration. Physical collaboration between two or more individuals involves both visual and haptic feedback. Here, we investigated how visual and haptic feedback is used to estimate the movements of a partner during a collaboration task. Our experimental and computational modeling results parsimoniously support the notion that greater visual accuracy is more important than faster yet noisier haptic feedback when estimating the state of a partner.