Hello everyone, today I would like to share a paper published in 2023 in Microsystems & Nanoengineering, titled “Computer vision meets microfluidics: a label-free method for high-throughput cell analysis”.
This study reviews the integration of microfluidic chips with computer vision, which has great potential in the analysis of single-cell imaging data. A key advantage of this integrated approach is the ability to achieve non-invasive and low-damage cell characterization. By combining Micro-Electro-Mechanical Systems (MEMS) technology with microfluidic chips and computer vision, it is expected to enable label-free, automated, low-cost, and rapid cell information recognition, as well as high-throughput analysis of cellular responses to different compounds, which has applications in drug discovery, diagnostics, and personalized medicine.
The authors of this paper are Professor Yan Hong’s team from Hainan University.
Background Introduction
With the rapid development of micro-nano manufacturing technology, Micro-Electro-Mechanical Systems (MEMS) and interdisciplinary research in life sciences, the manufacturing of microfluidic chips has gained increasing international attention in the biotechnology field. Microfluidic chips can integrate various cell detection devices such as electrodes, optics, and acoustics, and can flexibly design the structure and function of the chip as needed, not only meeting various types of cell analysis needs but also enabling the visualization of single-cell information. Furthermore, a series of operational steps in chemical and biological laboratories, such as sample preparation, reaction, separation, detection, cell culture, sorting, and lysis, have been miniaturized and integrated into a microfluidic chip to achieve high-throughput and high-speed detection. Cell analysis is a complex process involving various cell types and multiple biological responses in multidimensional space and time. Even under the same physical environment, culture conditions, and external stimuli, microbial cells from the same mother cell may exhibit significant differences in size, growth rate, morphology, and phenotype. Traditional cell detection methods, such as Enzyme-Linked Immunosorbent Assay (ELISA) and quantitative Polymerase Chain Reaction (qPCR), while providing unique insights into molecular biology and cell biology, often overlook important information regarding cell morphology and cell status, thus failing to elucidate biological phenomena or incidental events caused by cell interactions. Therefore, analysis of cell components, structure, and morphology at the single-cell level is crucial, as it not only helps explore cell heterogeneity but also uncovers interactions between cells and between cells and their environment, leading to a comprehensive understanding of the behavior of cells, tissues, organs, and even entire organisms.
Optical imaging has been the most direct way to observe microorganisms at the single-cell level since the mid-17th century when Antonie van Leeuwenhoek invented the microscope. Currently, microscopic observation has become a routine procedure for biologists and bioengineers to examine cell images and determine their viability. Common imaging methods include optical bright field microscopy, fluorescence microscopy, confocal laser scanning microscopy, super-resolution imaging, and lensless microscopy. Cellular or subcellular features are related to the structural and functional characteristics of biological systems and play an important role in cell identity recognition. Cell images contain a wealth of cellular information, such as size, morphology, texture, and internal structure. Compared to traditional univariate cell analysis methods, the main advantage of image-based cell analysis lies in its ability to achieve high-throughput screening of single-cell and multivariate imaging analysis. For example, in drug screening, visualized cell data can be interpreted to determine cell phenotypes, drug-induced gene expression, and protein localization, predict phenotypic changes in cytoskeletal structures, and achieve high-throughput screening of single-cell imaging.
Design Principles
Microfluidic chips can generate a large amount of visual data at the single-cell level, while computer vision technology can quickly process and analyze this data to extract valuable information about cell health and function. A key advantage of this integrated approach is the ability to achieve non-invasive and low-damage cell characterization, which is particularly important for studying fragile microbial cells.The article illustrates the comparison between the human visual system and computer vision in the cell recognition process through Figure 1, emphasizing the advantages of computer vision in handling large-scale data and complex image analysis, as well as its potential in simulating human visual processing.
Figure 1: Cell recognition through the human eye and brain versus computer vision.
Data Introduction
Applications of computer vision in cell analysis mainly involve four fundamental tasks: classification, localization, detection, and segmentation, as shown in Figure 2a. Since images are essentially matrices composed of pixel values, traditional computer vision methods can distinguish different categories of images by extracting features from the images (such as size, deformation, brightness, edges, texture, and color), allowing computers to “understand” the images. These features are typically expressed in numerical, vector, and symbolic forms. However, algorithm optimization and feature engineering for specific cells is a challenging task. Adjusted feature extraction algorithms are computationally efficient and fast, capable of handling high-throughput and relatively simple image processing tasks, but they fall short in addressing global feature variations and local contextual changes. In recent years, with the improvement of computational power and the development of artificial intelligence algorithms, neural networks have been widely applied in practice. Neural networks consist of multiple neurons, divided into input layers, hidden layers, and output layers, used for classification or prediction. To make neural networks more suitable for image analysis, researchers have proposed Convolutional Neural Networks (CNNs), whose architecture is inspired by the visual data processing mechanisms of the human visual cortex. Convolutional neural networks use convolutional kernels to process local image information in matrix form and are the most mainstream neural networks in the field of computer vision. Convolutional neural networks consist of convolutional layers containing multiple convolutional kernels, capable of learning features of cell images directly at the pixel level. The receptive field of the front-end convolutional layers is small, mainly capturing and computing local information and details of the image, while the back-end convolutional layers increase layer by layer to capture more complex and abstract image information (as shown in Figure 2b). Convolutional neural networks identify objects by extracting more abstract features, capable of capturing more variations and details of visual stimuli, and performing more complex cell image analysis tasks.
Figure 2: The four fundamental tasks in the field of computer vision.
Currently, the field of cell imaging in microfluidic cell analysis faces data processing bottlenecks, and computer vision algorithms, with their feature extraction and generalization capabilities, have become key solutions widely applied in cell characterization, detection, growth analysis, and more. In cell characterization, biophysical features (size, shape, mechanical properties, etc.) are crucial for revealing cellular homeostasis and disease mechanisms. Deformable cell techniques (such as contraction, shear flow, and stretching flow based cDC, sDC, xDC, as shown in Figure 3a) precisely control single cells flowing through detection areas using microfluidic chips, combined with traditional computer vision techniques (such as image acquisition, preprocessing, threshold segmentation, and feature extraction), utilizing the OpenCV library to efficiently extract spatial features such as cell size, roundness, and deformation, achieving high-throughput label-free detection, and have been applied in cancer research, blood testing, and other fields (as shown in Figure 3b). For example, Fregin et al. captured cell dynamics in real-time through sDC, calculating parameters such as apparent Young’s modulus, with a throughput of 100 cells per second (as shown in Figure 3c). Deep learning techniques do not require pre-defined morphological features, can uncover hidden information, explore phenotypic relationships, and cell heterogeneity. Although current detection throughput has not surpassed traditional methods, its potential in revealing biophysical biomarkers is enormous, pushing microfluidic cytometry towards intelligent analysis.
Figure 3: Microfluidic cytometry assisted by traditional computer vision algorithms.
Optical imaging systems on microfluidic platforms can non-destructively obtain internal structures and phenotypic features of cells, and their highly integrated characteristics make them ideal carriers for automatic analysis of cell images/videos using computer vision technology. The combination of the two enables cell detection, recognition, and sorting on chips: traditional feature engineering methods (such as threshold segmentation, watershed algorithms) can process cell images but are greatly affected by background noise; Convolutional Neural Networks (CNNs) do not require manual feature processing and are highly portable. For example, Nitta’s iIACS system integrates high-speed fluorescence microscopy with microfluidic chips to sort microalgae and blood cells in real-time; Chen et al. utilize FPGA for rapid signal transmission to reconstruct cell images; Godino manipulates cells through bright field imaging combined with dielectrophoresis; Nawaz achieves precise sorting of single cells using surface acoustic waves and deep neural networks; Girault’s label-free droplet sorting system sorts cells based on droplet morphological features. In cell detection and counting, Heo’s R-MOD system combines CNNs for real-time monitoring of microfluidic channels, while YOLO series algorithms are used for droplet and circulating tumor cell detection, supporting high-throughput analysis; lensless microscopy technologies (such as Göröcs’ holographic imaging system, Singh’s digital holographic microscope) break through lens limitations to achieve high-resolution imaging in large fields of view, with low cost and high portability, suitable for monitoring microbial communities and rare cell detection. The massive imaging data generated by microfluidic chips provides a training basis for neural networks, promoting research on the association between cell phenotypes and genotypes. With the improvement of algorithms and computational power, the intelligent application prospects of computer vision in microfluidic cell analysis are broad, as shown in Figures 4a-c, where various technical solutions have demonstrated their advantages in real-time detection, high-throughput sorting, and precise analysis.
Figure 4: Cell detection and classification based on computer vision technology on chips.
Unlike traditional petri dish methods, microfluidic technology can manipulate cells in a three-dimensional environment and maintain controlled growth, with its chip microchambers supporting high-throughput drug screening. However, large-scale applications are limited by the lack of rapid and low-cost readout methods. In recent years, computer vision methods have significantly broken through this bottleneck: Zhang et al. developed a three-dimensional tumor cell microfluidic platform (Figure 5a) that combines Convolutional Neural Networks (CNN) to automatically assess the activity of tumor spheroids and the efficacy of chemotherapy drugs through bright field image analysis; Su et al. utilized a microfluidic hydrogel droplet platform with CNN to classify crystal images (Figure 5b), efficiently screening conditions for indomethacin crystallization; Matsumoto and Yu’s system quickly detects bacterial drug sensitivity and dynamic phenotypic features through image analysis or deep learning algorithms (Figure 5c). Computer vision can not only quantify cell size, morphology, and movement in real-time across multiple dimensions but also handle ultra-large-scale high-resolution imaging data. Combined with microfluidic technology, it enables automated high-throughput analysis from three-dimensional cell culture to drug crystallization screening and antibiotic sensitivity testing, providing an unbiased and efficient screening tool for drug discovery.
Figure 5: Drug screening system based on computer vision.
The global pandemic has driven the demand for microfluidic POCT devices, with the core being accurate and sensitive signal readout. As shown in Figure 6, the nucleic acid aptamer barcode chip designed by Chen et al. achieves multiplex detection of exosomes through inexpensive imaging technologies such as Raspberry Pi; computer vision technology can accurately capture subtle color changes in chromogenic reactions to quantify target concentrations, and combined with lightweight convolutional neural networks, it can achieve real-time automated data processing on model devices (such as smartphones), suitable for remote areas. Although mobile imaging is limited by sensor parameters, denoising and resolution algorithms can enhance quality, and neural network-based visual models can reduce deployment costs with support from transfer learning, meeting complex cell analysis needs. The prospects for computer vision-based microfluidic POCT technology are broad, capable of integrating chips with mobile devices, simplifying healthcare, and improving clinical outcomes.
Figure 6: Computer vision-assisted rapid detection devices for on-site use.
Conclusion
This review article discusses the integration of microfluidic technology and computer vision in the life sciences and biotechnology fields, particularly in the analysis of single-cell imaging data. The article points out that microfluidic chips can generate a large amount of single-cell level visual data, while computer vision technology can quickly process and analyze this data to extract valuable information about cell health and function. A key advantage of this integrated approach is the ability to achieve non-invasive and low-damage cell characterization, which is particularly important for studying fragile microbial cells. Microfluidic chips provide a highly controllable environment for cell growth and operation, minimizing experimental variability and improving the accuracy of data analysis. Computer vision can be used to identify and analyze target species in heterogeneous microbial populations within complex biological systems, which is crucial for understanding the physiological states of cells in complex biological systems. With continuous improvements in hardware and artificial intelligence algorithms, computer vision is expected to become an increasingly powerful tool in on-site cell analysis. The combination of Micro-Electro-Mechanical Systems (MEMS) technology with microfluidic chips and computer vision is expected to enable label-free, automated, low-cost, and rapid cell information recognition, as well as high-throughput analysis of cellular responses to different compounds, which has broad application prospects in drug discovery, diagnostics, and personalized medicine.
Comments
1.This review article discusses the combination of microfluidic technology and computer vision in single-cell analysis, emphasizing the tremendous potential of this interdisciplinary approach in the life sciences and biotechnology fields. The article details how microfluidic chips generate large amounts of single-cell visual data and how computer vision technology quickly processes this data to extract information about cell health and function.
2.It emphasizes the advantages of computer vision in identifying and analyzing target species in heterogeneous microbial populations within complex biological systems. This method provides non-invasive and low-damage cell characterization, which is particularly important for studying fragile microbial cells. Additionally, the article discusses the highly controllable environment of microfluidic chips for cell growth and operation, and how computer vision technology minimizes experimental variability and improves the accuracy of data analysis.
Original Link
https://www.nature.com/articles/s41378-023-00562-8.pdfFOLLOW US
For submissions, recommendations, and collaborations, please contact us!
|| WeChat Official Account: Micro-Nano Sensing
|| WeChat ID: successplus999
Micro-Nano Sensor Industry Research
Academic Frontier Research Reports
Author: Tian Luyao
Editor: Teacher Zhang
Publisher: Tian Tiantian
Disclaimer ————
This article is for research sharing only and is not for profit. If there is any infringement, please contact the backend staff for deletion.
Due to limited knowledge, there may be omissions and errors. Please feel free to criticize and correct.