A public academic platform initiated by overseas scholars
Sharing information, integrating resources
Exchanging academic ideas, occasionally discussing other topics
Over 80% of the information processed by the human brain is obtained through the eyes. The human eye not only detects and synchronously processes information but also operates at an extremely low overall power consumption (far less than 20 watts). In contrast, traditional machine vision systems require detection followed by processing, where the image sensors generate a large amount of redundant information while detecting the target image. This information is transmitted to the connected computer for processing and analysis through limited bandwidth, resulting in significant time delays and higher power consumption. Therefore, constructing a brain-like visual sensor that can rival the human eye, capable of synchronously detecting and processing information, has been a long-standing dream, crucial for the development of applications in smart industry, autonomous driving, and intelligent security. Recently, Professor Miao Feng’s team from the School of Physics at Nanjing University has made significant progress in the field of reconfigurable brain-like visual sensors based on two-dimensional material heterojunctions.
Two-dimensional materials are expected to become important foundational electronic materials in the post-Moore era, allowing for the Lego-like stacking and integration of atomic-layer materials. In recent years, Professor Miao’s team at Nanjing University (https://nano.nju.edu.cn) has achieved breakthroughs in high-temperature memristors (Nature Electronics 2018), ballistic avalanche detector devices (Nature Nanotechnology 2019), and room-temperature high-sensitivity infrared detectors (Science Advances 2017) using “Atomic LEGO”.
Building on this foundation, the team recently proposed that by utilizing the structural characteristics and tunable optical response of two-dimensional van der Waals heterojunction devices, they can simulate the layered structure of the human retina and the biological characteristics of photoreceptor cells and bipolar cells. Based on this retina-like morphological device, the team further constructed a brain-like visual device array capable of synchronously processing perceived image information. This work is expected to provide a physical and technical foundation for the future development of novel brain-like visual chips based on van der Waals heterojunctions. The related research results were published on June 24, 2020, in Science Advances under the title “Gate tunable van der Waals heterostructure for reconfigurable neural network vision sensor“. PhD students Wang Chenyu and Associate Researcher Liang Shijun are co-first authors, while Professor Miao Feng and Professor Yang Jianhua from the University of Massachusetts are co-corresponding authors. This work also received experimental assistance from the research groups of Professors Wang Zhenlin, Chen Kunji, and Wang Xiaomu, as well as funding from the National Outstanding Youth Science Fund, the National Natural Science Foundation, and support from the Collaborative Innovation Center for Microstructure Science and Technology.
Research Results
The powerful information processing capability of the human visual system largely relies on the structure and function of the retina. The main cells in the retina include photoreceptor cells and bipolar cells, which are arranged in a vertically layered structure. When light passes through the pupil and strikes the retina, photoreceptor cells convert the incoming light into electrical signals, which flow through bipolar cells that process and refine the electrical information, retaining only the main features of the image before transmitting it to the cerebral cortex for further image processing and understanding. In this way, the retina achieves a certain degree of synchronous detection and processing of information. To realistically simulate the structure and function of the retina, the Miao team proposed constructing a brain-like visual sensor based on vertical heterojunctions of two-dimensional materials using the “Atomic LEGO” approach. These vertical structures not only naturally mimic the vertical layered structure of the retina but also allow different two-dimensional materials within the heterojunction to simulate the functions of different cells in the retina. In experiments, the Miao team first prepared vertical heterojunction devices from mechanically exfoliated thin layers of tungsten diselenide, boron nitride, and aluminum oxide. These heterojunction devices exhibited positive photoconductive behavior under no back gate or positive back gate voltage, similar to the positive light response of bipolar cells; when a negative back gate voltage was applied, the devices displayed negative photonic response characteristics akin to the negative response of bipolar cells. The research team pointed out through a series of comparative experimental results that the negative optical response of the devices arises from the electric field screening effect caused by light-induced charged impurities. By controlling the gate voltage of the vertical heterojunction devices, the team achieved, for the first time, the simulation of the biological functions of photoreceptor cells and bipolar cells, with the device’s response time and power consumption approaching that of the human retina (Figure 1).
Figure 1. Retina and retina-like morphological devices. (a) The vertical layered structure of the retina includes photoreceptor cells, bipolar cells, and ganglion cells; (b) The biological optical response characteristics of bipolar cells under different stimulation conditions; (c) Optical image of the retina-like morphological device; (d) The vertical heterojunction device exhibits different optical responses under different gate voltages, similar to the biological characteristics of bipolar cells in the retinal structure. Furthermore, the research team assembled the vertical heterojunction devices into a 3×3 array, utilizing the tunable photonic response characteristics of the heterojunction devices to map commonly used mathematical convolution kernels in image processing to the 3×3 device array, achieving reconfigurable image information processing functions (Figure 2), including edge enhancement, image stylization, and intensity correction. The experimental results were found to be consistent with the simulated results obtained using the same convolution kernels, indicating that the vertical heterojunction array can be used for direct hardware processing of image information.
Figure 2. Reconfigurable retina-like chip and its various image information processing methods. (a) Schematic diagram of the reconfigurable visual sensor array with independent control of the back gate; using the sensor array to process the Nanjing University emblem image for (b) image stylization, (c) edge enhancement, and (d) intensity correction, with the processed image information consistent with the simulated results using the same convolution kernels. In addition to synchronous detection and information processing, the research team found that the device array can also be used for image classification tasks. By using the device conductance as the weights of a neural network and the adjustments of the back gate as a means to update the neural network weights, the van der Waals heterojunction device array can perform neural network functions. The research team implemented rapid recognition of input images of the letters “N”, “J”, and “U” using a software-assisted hardware training method (Figure 3). This work fundamentally demonstrates that the research approach of simulating the structure and function of the human retina using the characteristics of van der Waals heterojunctions holds promise for the future development of novel brain-like visual chips.
Figure 3. Schematic diagram and recognition results of the sensor neural network based on van der Waals heterojunction visual morphology. (a) The image letters to be recognized; (b) Schematic diagram of the training of the heterojunction visual morphology sensor neural network; (c) Average recognition results of the letters “N”, “J”, and “U” in relation to the number of training iterations; (d) Classification of the letters “N”, “J”, and “U” in relation to the number of training iterations.
Click below to Zhisheng Talent Square to view the latest academic recruitment.
Click “Read the original text” to view the original paper.
Further Reading
Goodbye to batteries, biofuel-powered electronic skin for real-time health monitoring and human-machine interaction
Peking University SH ultrasonic guided wave transducer, opening a new era of structural health monitoring
Science fiction approaches reality, Caltech micro-machines for real-time imaging, regulation, and treatment of diseases
This article is a feature content of NetEase News · NetEase Account “Each has its own attitude”
For media reprint authorization, please see below
