A Silver Nanowires‐Based Flexible Capacitive Touch Screen in Tactile Displays for Individuals with Visual Impairment Using Gesture Recognition

• Published in: Advanced materials technologies 2024-11, Vol.9 (24), p.n/a
• Main Authors: Hamza, Ahmed, Alzalabny, Sara, Buduru, Priyanka, Bhagwat, Sagar, Usama, Ali, Prabhulingaiah, Santosh Kumar, Song, Qingchuan, Kluck, Sebastian, Jaworek, Gerhard, Pezeshkpour, Pegah, Rapp, Bastian E.
• Format: Article
• Language: English
• Subjects: capacitive touch screens; hand gesture recognition; machine learning; silver nanowires; tactile displays; visually impaired equity
• ISSN: 2365-709X; 2365-709X
• DOI: 10.1002/admt.202401029

Capacitive touch screens (CTS's) are essential components in most of today's digital devices. However, for the visually impaired (VI) users due to the uneven topography of the tactile surface, CTS's are more challenging to implement and thus this field remains largely underdeveloped. Considering the limited space around the microactuators driving the typical Braille dots for a tactile screen with ten dots‐per‐inch (dpi) resolution, the materials used for CTS should be flexible and durable with high mechanical strength. In this work, a flexible CTS based on polyimide (PI) and silver nanowires (AgNWs) as electrodes with a total thickness of 210 µm is developed. The dimensions of the AgNWs are on average 7.9 ± 2.4 µm in length and 85 ± 24 nm in width. The AgNWs electrodes showed low resistance and good adhesion to the PI substrate. A gesture recognition application is collected from the capacitive data to classify different gestures (including single‐ and double‐click, swipe‐left and ‐right, scroll‐up and ‐down as well as zoom‐in and ‐out) with two different approaches; machine learning and deep learning are implemented. The best performance is obtained using the YOLO model with a high validation accuracy of 97.76%. Finally, a software application is developed with the proposed hand gestures in real‐time to foster interaction of VI users with the tactile display allowing them to navigate a Windows file system and interact with the documents via hand gestures in a similar manner as sighted users on a conventional touch display will be able to do. This work paves the way to utilize CTS for the tactile displays in the market developed for VI users. A capacitive touch screen (CTS) is proposed here using hand gesture recognition for integration with Braille tactile displays for individuals with visual impairment. The CTS consists of a flexible polyimide layer and silver nanowire electrodes. The data are collected to classify several gestures using machine learning and deep learning. Using the YOLO model, a high validation accuracy, 97.76%, is achieved.