Mitigation of Accuracy Degradation in 3D Flash Memory Based Approximate Nearest Neighbor Search with Binary Tree Balanced Soft Clustering for Retrieval-Augmented AI

• Main Authors: Sasaki, Shinichi, Aiba, Yuta, Komano, Yusuke, Iizuka, Takahiko, Fujimatsu, Motohiko, Kawasumi, Atsushi, Miyashita, Daisuke, Deguchi, Jun, Maeda, Takashi, Miyano, Shinji, Maruyama, Tooru
• Format: Conference Proceeding
• Language: English
• Subjects: 3D Flash Memory; Accuracy; Approximate Nearest Neighbor Search; Binary trees; Clustering; Common Information Model (computing); Computing-in-memory; Degradation; In-memory computing; Nearest neighbor methods; Three-dimensional displays
• ISSN: 2474-9672
• DOI: 10.1109/NewCAS58973.2024.10666332

Computing-in-memory (CIM) can suppress the energy consumed in transferring data from the memory to the processor. However, implementation of CIMs is often impractical due to the overhead of adding circuits on memory to perform high-precision processing. In this paper, we propose techniques to realize CIM using a standard 3D flash memory for approximate nearest neighbor search (ANNS), and demonstrate their application to retrieval-augmented AI. To maintain high computational efficiency, we introduce binary tree balanced soft clustering, which divides the dataset into the page size while avoiding accuracy degradation. Furthermore, we introduce similarity distribution learning, which focuses on the tail of the distribution to prevent accuracy degradation in data transformation for CIM. We conducted ImageNet-1k image classification experiments as an evaluation of retrieval-augmented AI including ANNS, and achieved top-1 accuracy of 77.7%, which is comparable to digital processing.