1
Overview
——A New Breakthrough in Human Visual Biomimicry: Ultra-Adaptive Neuromorphic Devices and Fully Homogeneous General Intelligent Visual Systems
Visual events in nature exhibit multi-dynamic and unpredictable characteristics, posing significant challenges to traditional machine vision systems. In this context, Artificial General Visual Intelligence (AGVI) emerges as a new paradigm aimed at replicating the full-spectrum dynamic characteristics of the human visual system within a single hardware system, achieving superior energy efficiency and scene adaptability. Its core lies in the multimodal signal mechanism that simultaneously mimics the retinal and cortical neurons, incorporating both “all-or-nothing” action potentials and graded potentials proportional to stimuli, while needing to accommodate both optical and electrical dual-dimensional control.
However, the current cutting-edge AGVI solutions are still primarily based on CMOS pathways. Although theoretically, various neuromorphic components with higher-order, rich dynamics can be further developed along the developmental path and AGVI systems can be constructed through heterogeneous integration, such systems still struggle to fundamentally overcome the inherent contradictions between flexibility, energy efficiency, and volume when faced with continuously changing real-world application scenarios. More importantly, such heterogeneous systems have not yet been realized, leaving AGVI development in an early exploratory stage. Therefore, the key to achieving high-performance AGVI lies in transcending the heterogeneous integration phase, integrating the core complex dynamics of the retina and cortex into a single electronic component, thus promoting the system architecture from “heterogeneous integration” to a “highly homogeneous” paradigm, thereby unifying neuromorphic dynamics at the device level and addressing the fundamental issue of balancing energy efficiency and adaptability. However, this leapfrogging path faces fundamental challenges, as no single electronic device has yet been able to simultaneously integrate retinal-like and cortical-like pulse and graded potentials at the intrinsic level, and achieve programmable operations across optical and electrical domains. This presents unprecedented demands on the device’s multidimensional signal mechanisms and adjustable electronic characteristics, becoming a core technical challenge that must be overcome to realize a fully homogeneous AGVI system.
To this end, Professor Zhou Feichi’s team from the Southern University of Science and Technology’s Shenzhen-Hong Kong Institute of Microelectronics, in collaboration with Beijing Xiling Visual Technology Company and Professor Cai Songhua’s team from the Hong Kong Polytechnic University, has successfully developed an ultra-adaptive neuromorphic visual device that is currently the closest to human vision, achieving the first homogeneous integration of key neuronal components of the retina and cortex. In a single device (IxTyO1-x-y/CuOx/Pd), it realizes the full functional integration of four types of high-order dynamics: a wide-spectrum (375-1064 nm) tunable retinal-like optical pulse neuron (RSN), a retinal-like optical graded neuron (RGN), and electrically tunable cortical-like synapses (CS) and cortical-like neurons (CN), while supporting cross-paradigm computing – pulse/non-pulse/in-memory computing/storage computing. Notably, the RSN mode is the first to be realized in a single device and operates under zero bias and without external capacitance, demonstrating superior integration, wide spectral response, and ultra-low power consumption compared to current multi-device integration solutions.
Figure 1. Ultra-Adaptive Neuromorphic Visual Device (UANV).
This single device perfectly simulates the multi-order dynamics from perception to computation in biological visual pathways, highly integrating functions that would traditionally require over 60 discrete components in CMOS solutions. This rich and controllable high-order dynamic characteristic stems from a unique energy band structure and the synergistic control of charge carriers (electrons, oxygen ions, and vacancies) at the bulk or interface under optical/electrical stimulation. More importantly, the team, by combining in-situ scanning transmission electron microscopy (STEM) and TCAD simulations, has for the first time deeply revealed the high-order dynamic mechanisms of neuromorphic visual devices, providing a crucial theoretical foundation and design guidelines for the design of integrated neuromorphic visual devices for perception, storage, and computation (Figure 2).
Figure 2. Physical Mechanisms of Four Dynamic Modes in UANV Device.
The successful development of this device breaks through the functional limits of traditional oxide semiconductor devices, highly integrating visual intelligence functions that previously relied on complex circuits into a single micro-device. This achievement not only fills the gap in the mechanistic explanation of high-order dynamic neuromorphic visual devices but also demonstrates the enormous potential of integrated perception, storage, and computation devices in large-scale manufacturing, providing a core hardware foundation for constructing truly compact, efficient, and adaptive artificial visual systems. Furthermore, this device fundamentally resolves the long-standing contradiction between system flexibility and energy efficiency, promoting the transition of AGVI from “heterogeneous integration” to “fully homogeneous integration”. This lays a new hardware paradigm and feasible technical path for developing the next generation of minimalist, ultra-adaptive, and high-energy-efficient general intelligent visual systems.
Furthermore, the team has constructed an “ultra-adaptive” AGVI hardware system and prototype platform (Figure 3). This system is based on an array of fully homogeneous neuromorphic visual devices and dynamic control circuits, capable of flexibly allocating the dynamic modes of the array according to scene requirements, switching flexibly between pulse, non-pulse, in-memory, and storage computing modes. This design allows a single hardware platform to adapt to multi-scene processing and supports flexible reconstruction of various paradigms such as event-driven, frame-driven, neuromorphic perception, and cognitive computing, thus achieving optimal performance for different tasks. The system achieves an energy efficiency of up to 67.89 TOPS/W, with an area efficiency of 3.96 MOPS/F, surpassing traditional AGVI systems by 2-3 orders of magnitude.
Figure 3. “Ultra-Adaptive” AGVI Hardware System and Prototype Platform Based on Fully Homogeneous UANV Devices.
The related results were published in Nature Nanotechnology under the title “High-order dynamics in an ultra-adaptive neuromorphic vision device”. Professor Zhou Feichi from Southern University of Science and Technology and Professor Cai Songhua from Hong Kong Polytechnic University are co-corresponding authors, while master’s student Xu Jiayi, doctoral student Jiang Biyi from Southern University of Science and Technology, and postdoctoral researcher Wang Weizhen from Hong Kong Polytechnic University are co-first authors. This research was supported by the National Key R&D Program and the National Natural Science Foundation.
2
Main Author Introduction
Corresponding Author
Zhou Feichi, researcher/PhD supervisor at the School of Microelectronics, Southern University of Science and Technology, selected as a national-level overseas high-level talent (youth project), chief scientist of the National Key R&D Program youth project, recipient of the Hong Kong Young Scientist Award, and overseas high-level talent in Shenzhen.
He has long been engaged in research on integrated neuromorphic devices, artificial intelligence visual chips, post-Moore new semiconductor devices, and advanced three-dimensional monolithic integrated perception and computation chips. Related research results have been published in over 60 papers as first/corresponding author in journals such as Nature Nanotechnology (2 papers, 1 ESI highly cited), Nature Electronics (ESI highly cited), Nature Communications (2 papers), Advanced Materials, etc. As the project leader, he has presided over the National Key R&D Program youth scientist project, National Natural Science Foundation overseas outstanding youth project, National Natural Science Foundation general project, and youth fund projects. He established the Shenzhen-Hong Kong Institute of Microelectronics – Xiling Visual Neuromorphic Perception and Computation Chip Joint Laboratory, serving as the laboratory director. He serves as a youth editorial board member for journals such as InfoMat, Nano-Micro Letters, and Materials Futures.
Corresponding Author
Cai Songhua, assistant professor in the Department of Applied Physics at Hong Kong Polytechnic University.
He is mainly engaged in the development and application research of advanced transmission electron microscopy and in-situ techniques, having published a total of 24 papers as first/corresponding author (including co-authored) in internationally renowned academic journals such as Nature, Nature Electronics, Nature Nanotechnology, Nature Synthesis, Nature Communications, Advanced Materials, JACS, etc., obtained three Chinese invention patents, applied for one US invention patent, awarded the 2023 Hong Kong Polytechnic University Young Innovator Award, and presided over projects such as the Hong Kong Research Grants Council General Research Fund (GRF) and the Outstanding Young Scholars Scheme (ECS), as well as the National Natural Science Foundation youth fund project, serving as a youth editorial board member for the journal Microstructures.
Xiling Vision is an industry-leading technology chip company dedicated to developing a new generation of intelligent visual sensor chips. Xiling Vision’s pixel-level perception and computation integration technology, after years of accumulation and iteration, surpasses traditional von Neumann architecture processing methods, fundamentally creating technological innovations that efficiently combine pixel and computation technologies, bringing unparalleled advantages in speed, low power consumption, and intelligence compared to traditional architectures. The company’s technology has been widely applied in commercial scenarios such as machine vision, embodied intelligence, AR/VR, and drones. In the future, it will also serve as a key bridging technology for optoelectronic integrated computing, connecting light and electricity, analog and digital, perception and intelligence. The company’s first mass-produced product won the 18th “China Chip” Chip Fire New Product Award in 2023.
3
Original Transmission
For details, please click the paper link:
https://www.nature.com/articles/s41565-025-01984-3
Introduction to the Journal of Semiconductors (English):
The Journal of Semiconductors (English) is an academic journal supervised by the Chinese Academy of Sciences, co-sponsored by the Institute of Semiconductors, Chinese Academy of Sciences, and the Chinese Institute of Electronics, founded in 1980, with the first editor-in-chief being Academician Wang Shouwu, and Mr. Huang Kun writing the first paper in the inaugural issue. In 2009, it was changed to an all-English journal, Journal of Semiconductors (abbreviated as JOS), and began cooperating with the IOPP British Institute of Physics Publishing to distribute globally. The current editor-in-chief is Academician Li Shushen. It has won the title of China’s most internationally influential academic journal for ten consecutive years. The impact factor is 5.3, ranking 22/79 in the field of condensed matter physics, at the top of Q2 zone.
“Half-Speak – Beneficial Words” Lecture Series
Using a few words to discuss “core” technology, the “Half-Speak – Beneficial Words” five-season live lecture replay link:
https://www.koushare.com/topicReview/byyy/68
The sixth season of the 2025 live lecture will be held irregularly, please stay tuned.
Introduction to the Recommendation and Selection Work of the Top Ten Research Advances in Semiconductors:
The Journal of Semiconductors (English) launched the recommendation and selection work for the “Top Ten Research Advances in Semiconductors” at the beginning of 2020, recording landmark achievements in the field of semiconductor science and technology research in China. Research results published this year by institutions such as research institutes, universities, and enterprises in China can participate in the selection. Recommend or self-recommenders should send the PDF file of the research results to the Journal of Semiconductors (English) email: [email protected], along with a brief recommendation reason. The recommended person must provide a work summary of about 500 words, explaining the academic value and application prospects of the research results. The annual top ten research advances will be voted on by the expert review committee from the recommended results and announced before the next year’s Spring Festival.
Introduction to the JOSarXiv Preprint Platform:
With the rapid development of semiconductor technology, the number of scientific papers produced has been increasing year by year. JOSarXiv is dedicated to providing a platform for domestic and international semiconductor researchers to publish and access Chinese and English scientific papers for free, ensuring the recognition of the first publication rights of excellent research results and promoting broader academic exchanges. JOSarXiv was established under the advocacy of Academician Li Shushen, editor-in-chief of the Journal of Semiconductors (English), and is managed by the editorial office, being the first dedicated preprint platform for the semiconductor technology field in China, providing services for preprint paper storage, retrieval, publication, and sharing. JOSarXiv officially launched on January 1, 2020 (http://arxiv.jos.ac.cn/), and can also be accessed through the Journal of Semiconductors (English) official website (http://www.jos.ac.cn/). Please pay attention and submit!