Camera Frame Misalignment in a Teleoperated Eye-in-Hand Robot: Effects and a Simple Correction Method

• Published in: IEEE transactions on human-machine systems 2023-02, Vol.53 (1), p.2-12
• Main Authors: Wu, Liao, Yu, Fangwen, Do, Thanh Nho, Wang, Jiaole
• Format: Article
• Language: English
• Subjects: Camera frame misalignment; Cameras; Completion time; Confined spaces; Electron tubes; End effectors; eye-in-hand; human–robot interaction; Input devices; Misalignment; Pick and place tasks; Psychological stress; Rigid structures; Robot vision systems; Robots; teleoperation; Visualization
• ISSN: 2168-2291; 2168-2305
• DOI: 10.1109/THMS.2022.3217453

Misalignment between the camera frame and the operator frame is commonly seen in a teleoperated system and usually degrades the operation performance. The effects of such misalignment have not been fully investigated for eye-in-hand systems-systems that have the camera ( eye ) mounted to the end-effector ( hand ) to gain compactness in confined spaces such as in endoscopic surgery. This article provides a systematic study on the effects of the camera frame misalignment in a teleoperated eye-in-hand robot and proposes a simple correction method in the view display. A simulation is designed to compare the effects of the misalignment under different conditions. Users are asked to move a rigid body from its initial position to the specified target position via teleoperation, with different levels of misalignment simulated. It is found that misalignment between the input motion and the output view is much more difficult to compensate by the operators when it is in the orthogonal direction (\sim40 s) compared with the opposite direction (\sim20 s). An experiment on a real concentric tube robot with an eye-in-hand configuration is also conducted. Users are asked to telemanipulate the robot to complete a pick-and-place task. Results show that with the correction enabled, there is a significant improvement in the operation performance in terms of completion time (mean 40.6%, median 38.6%), trajectory length (mean 34.3%, median 28.1%), difficulty (50.5%), unsteadiness (49.4%), and mental stress (60.9%).