Predicting Real‐Time Spectra from High‐Speed Imaging: An Ultrafast Machine Vision Framework for Online Optical Control in Microfluidics

• Published in: Advanced materials technologies 2024-08, Vol.9 (15), p.n/a
• Main Authors: Zhang, Xucheng, Guo, Jia, Wang, Changchun
• Format: Article
• Language: English
• Subjects: deep learning; machine vision; microfluidics; self‐driving laboratories; spectroscopy
• ISSN: 2365-709X; 2365-709X
• DOI: 10.1002/admt.202101344

Automated laboratory pipelines empowered by the advent of artificial intelligence (AI) have been reshaping the paradigm of chemical and material discovery. While intelligent decision‐making strategies have been developed that enable fast convergence to optimal reaction conditions, the overall efficiency is still bottlenecked by the slow acquisition of online characterization units, especially in microfluidics‐based systems. In response, an ultrafast machine vision framework is hereby developed to directly predict optical spectra from real‐time high‐speed imaging in this article. Utilizing uniquely engineered deep‐learning algorithms, this framework automatically detects droplets and recapitulates high‐quality spectra from embedded image features in a sub‐millisecond timeframe. In a proof‐of‐concept example, droplets are detected with an average precision of 99.75% within 0.51 ms, and their spectra spanning the entire visible wavelengths are predicted with an average peak error of 5.32 nm, which is equivalent to 1.6% of the visible range. The work demonstrates the first example of an all‐imaging‐based machine learning framework deployed in a real‐time environment for the full replacement of an analytical instrument. It highlights the broader possibility of AI in material researches to break the physical limitations of online diagnostics and to further boost efficiencies in self‐driving laboratories. The machine vision framework automatically detects droplets from the high‐speed camera stream in real‐time and predicts optical spectra from the images. The framework features fully automated, spatially resolved, ultrafast optical tracking of the droplets. The modular design based on robot operating system allows easy integration of even more functionalities to facilitate applications such as self‐driving laboratories.