Hello everyone! Long time no see! I have some exciting news to share with you all. Our team members are currently organizing it, and it will be announced soon! Thank you!
A few days ago, a friend pointed out some inaccuracies in the text regarding robots. I am very grateful for their careful review; this is the spirit of science and the pursuit of truth! So, I am reposting it and also organizing links to other content about robots at the end of this article. Thank you again for your support!
Lecture 3: Is Gradient-Free Really Possible?
——How to Achieve Completely “Structure-Driven” AI Training?Keywords:Cannistraci-Hebb Training, Topology-Aware Learning, Gradient-Free Evolution, Network Science, Boundary Integration Principle
Since I proposedCHT (Cannistraci-Hebb Training), the most common question I get is:
“Professor, you said it does not rely on gradient training, so how does it actually ‘learn’?”
This is a good question. After all, in today’s context of AI, learning is almost synonymous with “backpropagation.” Without gradients, how can we train a network? Without backpropagating errors, how do we know where we went wrong?
Today, I will systematically answer this question:
Gradient-free is not only possible but alsocloser to the human brain, more robust and efficient, and better suited for the learning needs of embodied robots.
We need to delve intoCHT‘s core mechanism—a framework for building intelligence that completely breaks free from gradients and relies onstructural changes and principles of network science.
1. Change the question: How is the human brain trained?
In the brain, there is no “loss function,” no “error derivative.” Children learn to grasp objects, recognize shapes, and understand emotions solely through the repeated activation, connection, and disconnection of neurons. As theHebbian learning rule states:
“Cells that fire together, wire together.”——Donald Hebb, 1949
This learning mechanismdoes not pursue error minimization but is based on synchronous activity and structural reorganization.
In other words:
Intelligence does not come from gradient guidance but from selective activation and memory of structures.
2. The basic mechanism ofCHT: Activation → Resonance → Growth of Connections
CHT adopts principles similar to biological neurons for “structural learning.”
- Forward Activation Propagation: Input signals propagate from the input layer through the network, forming a set of activated nodes;
- Co-activation Detection: Identifying pairs of neurons with co-activation relationships among the activated nodes;
- Hebbian Structural Update: Strengthening connections, adding new connections, or weakening inactive connections based on topological rules;
- Network Update and Sparsity Maintenance: Maintaining a certain level of sparsity in the network hierarchy to avoid overfitting or redundancy.
No backpropagation, no gradient calculation, no reliance on label supervision is needed during the network evolution process; training the network relies solely on activation states and structural evolution.
3. Core Theory 1: Topology-Aware Structural Evolution
I introduced principles of complex systems and network science into neural network training, forming a significant feature ofCHT.
Specifically:
- The generation, retention, or deletion of connections between neurons is not random, nor based on errors, but according totopological metrics (such as node centrality, path embedding, boundary attraction, etc.);
- The system will prioritize formingpaths with high information transmission efficiency;
- Brain-inspired structures (such as small-world properties, modularity, low-degree distribution) emerge spontaneously.
This structural evolution mechanism allowsCHT to findthe most effective information flow channels even without guidance from a “target function.”
The intuitive analogy is as follows:
If gradient learning is like climbing a mountain—constantly adjusting towards the direction of the lowest error, thenCHT is like building a city traffic network—growing roads based on human flow rather than having planners draw all routes in one stroke.
4. Core Theory 2: Boundary Integration Principle
In my early research on network science and complex systems, I proposed an important principle:
The most effective information convergence points in a network often lie inboundary areas, which are the junctions of different modules.
This principle has been incorporated intoCHT, transforming it into a dynamic connection mechanism:
- CHT will identify pairs of neurons that are concentrated in activation but have different modular structures;
- Connections will preferentially grow at these “structural boundaries”;
- Forminglow-density but cross-modal highly cooperative bridge paths.
This is similar to how “cross-modal integration” (such as between speech and images, touch and vision) occurs in boundary areas (like the association cortex, insular region) in the human brain—we have also simulated a similar mechanism for robots.
5. Systematic Differences Between CHT Structural Evolution Paradigm and Gradient Training
|
Dimension |
Gradient Training |
CHT Structural Evolution |
|
Core of Learning |
Parameter Tuning |
Connection Structure Evolution |
|
Signal Source |
Error Derivative |
Node Co-activation |
|
Optimization Target |
Loss Function |
Network Topology Structure |
|
Dependent Information |
Label Supervision |
No Labels, No Gradients |
|
Model Update |
Global Backpropagation |
Local Plasticity |
|
Learning Path |
Single Target Direction |
Diverse Topological Reconstruction |
|
Deployment Cost |
High Computing Power/Power Consumption |
Extremely Low Power Consumption, Sparse Computing |
The conclusion is clear:
CHT is a learning method that starts from “connection shaping,” not from “error correction.”
6. A Shift in Thinking: From “Tuning Parameters” to “Growing Structures”
The greatest challenge posed byCHT is not mathematical complexity butthe disruption of cognitive paradigms.
In the old paradigm, we always thought of “training” as “finding a good set of parameters.” However, inCHT, “training” becomes “generating a good structure.”
It is like gardening, not trimming branches and leaves, but guiding trees to naturally grow into forms that fit the environment; it is like urban expansion, not overall design, but allowing traffic, population, and geography to organically give rise to optimal forms.
This is also why I firmly believeCHT is suitable for robots:
True embodied intelligence is not an art of parameter setting but a science of structural growth.
Collection of Five Lectures on Robots:
Scientific Exploration | Tsinghua Professor Discusses AI—Special Five Lectures on Robots: Why Sparse Algorithms are the “Soul Structure” of Next-Generation Robot Intelligence?
Scientific Exploration | Tsinghua Professor Discusses AI—Special Five Lectures on Robots: Why Do Robots Need “Sparse Intelligence”?
Scientific Exploration | Tsinghua Professor Discusses AI—Special Five Lectures on Robots: Why Do Robots Learn Poorly? Is it the Fault of “Gradient Training”?
Scientific Exploration | Tsinghua Professor Discusses AI—Special Five Lectures on Robots: Can Sparse Algorithms Enable Robots to Operate “Like Life”?
Scientific Exploration | Tsinghua Professor Discusses AI—Special Five Lectures on Robots: The Future Robots are Not Programmed but “Evolved”!
If you are interested in my research, feel free to follow the “Kai Long’s Sparse Algorithm Laboratory” public account, and let’s explore the future of AI together! 🚀🚀🚀