According to a report by MEMS Consulting, the team led by Fan Zhen from the South China Advanced Optoelectronics Research Institute has recently developed an integrated ferroelectric photovoltaic sensor network, achieving zero-energy image perception and real-time processing. This provides a feasible technical route for developing highly robust, high-speed, and low-energy machine vision systems. The related results were published in Nature Communications under the title “Ferroelectric photosensor network: an advanced hardware solution to real-time machine vision.” Master’s student Cui Boyuan is the first author of the paper, with Researcher Fan Zhen as the corresponding author.
Machine vision is a technology that endows machines with the ability to “see” and “cognition,” widely applied in daily life, such as facial recognition, smart manufacturing, and medical imaging diagnostics. Traditional machine vision systems usually collect data through sensors like cameras and then transmit it to processors for processing. The repeated transportation of data between sensing, memory, and processing units leads to high energy consumption and latency, making it difficult to meet the real-time processing demands of massive visual information in scenarios like autonomous driving. In contrast, the human visual system can quickly complete complex visual cognition with extremely low energy consumption. Thus, the concept of brain-like vision has been proposed and quickly became a research hotspot. Brain-like vision systems adopt near-sensing or in-sensor computing architectures, significantly reducing data transportation between functional units, thereby greatly enhancing the speed and energy efficiency of visual information processing. Many hardware implementations of brain-like vision have been reported, among which tunable optoelectronic sensor networks have attracted attention due to their simple structure and integrated sensing-computing capabilities. It utilizes the optical responsivity of its structural units (i.e., tunable optoelectronic sensors) to represent synaptic weights, achieving the embedding of artificial neural network algorithms, thus enabling image perception and in-situ processing (including recognition, detection, and other advanced processing functions).
Currently, the construction of tunable optoelectronic sensors mainly relies on two-dimensional materials and memristive materials. The former controls optical responsivity using a volatile gate control effect, but continuous application of gate voltage can lead to additional energy consumption and requires extra storage units to save weights; the latter controls optical responsivity using ion migration, but this process is relatively slow and has relaxation issues. Therefore, developing tunable optoelectronic sensors based on new materials and principles that are highly robust, high-speed, and low-energy is of significant importance, thereby constructing an integrated sensing-computing sensor network.
Real-time Image Perception + Edge Detection
The Fan Zhen team first utilized the large polarization and polarization-controlled photovoltaic effect of the ferroelectric Pb(Zr0.2Ti0.8)O3 epitaxial thin film to construct high-quality ferroelectric photovoltaic sensors, achieving multi-level, non-volatile, positive-negative symmetrical tunable photocurrent. Then, multiple ferroelectric photovoltaic sensors were assembled into an integrated sensing-computing artificial neural network circuit, using optical responsivity to represent weights, proving the feasibility of in-situ multiplication-accumulation operations and achieving zero-energy image perception and real-time processing functions. The highlights and features of this research include: (1) Ferroelectric photovoltaic sensors utilize non-volatile polarization to control optical responsivity (weights), eliminating the need for continuous external voltage to maintain weights. (2) Ferroelectric photovoltaic sensors exhibit positive-negative symmetrical tunability of optical responsivity, thus requiring only a single device to represent both positive and negative weights. (3) Under a constant polarization state, the output photocurrent of the ferroelectric photovoltaic sensor maintains a linear relationship with the input optical power, ensuring high accuracy in multiplication operations. (4) In the inference process, each device of the ferroelectric photovoltaic sensor network operates under short-circuit conditions, with zero energy consumption during inference. Additionally, short-circuit conditions and extremely high dark-field resistance minimize the current contribution of unilluminated devices, resolving crosstalk issues and ensuring high accuracy in multiplication-accumulation operations. (5) The ferroelectric photovoltaic sensor network achieved an accuracy of up to 100% in binary image perception and recognition tasks, and an F-Measure of up to 1 in custom image perception and edge detection tasks. (6) Utilizing the ultra-fast photocurrent generation process and integrated sensing-computing architecture, the ferroelectric photovoltaic sensor network can achieve ultra-low latency. The paper link: https://www.nature.com/articles/s41467-022-29364-8Further Reading: “Emerging Image Sensor Technologies, Applications, and Markets – 2021 Edition” 