Image source:Daniel Picard
Leviathan Note: Rather than saying that the robots in Professor Lipson’s lab have “achieved a form of self-awareness,” it is more accurate to say that these robots demonstrate a series of mechanisms regarding information input and output, aside from the “source” of self-awareness. For practical research purposes, such models can indeed be referred to as a form of “self-awareness.” While philosophical discussions about the origins of self-awareness have long been mired in difficulties, it is encouraging to see scientists using robots to reverse-engineer the birth of consciousness.
Written by/John PavlusTranslated by/Trion BingProofread by/He Li HuoOriginal text/www.quantamagazine.org/hod-lipson-is-building-self-aware-robots-20190711This article is published by Trion Bing on Leviathan under a Creative Commons license (BY-NC).The views expressed in this article are those of the author and do not necessarily represent the position of Leviathan.“I really hope to see aliens in my lifetime!” said Hod Lipson, director of the Creative Machines Lab at Columbia University, who also expressed a desire to “see some form of intelligent non-human entity before I die.” Rather than waiting for intelligent life forms to descend from the sky, Lipson decided to create one himself in the lab; to be precise, he is working on building machines with self-awareness.To achieve this goal, Lipson must confront the nebulous concept of consciousness.However, the situation he faces is not optimistic; even Lipson’s colleagues avoid discussing this concept, treating it as taboo. “In the fields of robotics and artificial intelligence, humanity has long regarded consciousness as a taboo topic that no one can touch,” Lipson admitted, “because the concept is too ambiguous, and no one knows what consciousness really is. But since we are doing rigorous scientific research, we must confront this concept. Unfortunately, as far as I know, what consciousness really is can be considered the greatest unsolved mystery, comparable only to the origins of life and the universe.”Think about it—what is feeling? What is creativity? What are emotions? Today, humanity not only wants to know what humanity is but also hopes to understand how to artificially create humanity. Well, it is time to face these questions; we can no longer shrink back from them.”
Hod Lipson in the Creative Machines Lab at Columbia University. Image source: QuantaAccording to Lipson’s research,the most important cornerstone of self-awareness is “self-simulation”: creating a virtual model of the body that describes how the body moves in real space and guides actions through this model.Since early 2006, Lipson has been researching artificial self-awareness, starting with the creation of a sinister-looking spider-like robot that learned to move in a straight line across a table using an evolutionary algorithm.(As a hint, in addition to the evolutionary algorithm, Lipson also pre-installed some programs to explain basic physical rules.)
In a TED talk, Lipson demonstrated how the spider-like robot learned to crawl on its own. Initially, these robots would swing their limbs aimlessly, attempting to establish their virtual model based on the information collected. The image above shows the spider-like robot finally completing its self-modeling and simulating a crawling motion based on this model. Image source: TED
“Look at this guy; he is still far from conquering the world,” Lipson said while showing a video of the spider-like robot in action. Although the crawling motion looks both sinister and clumsy, the robot has taken a small step entirely based on its learning ability. Image source: TED(www.ted.com/talks/hod_lipson_builds_self_aware_robots?language=en#t-8208)By 2012, the research on creating artificial intelligence using modern technology finally gained momentum, with convolutional neural networks (CNNs) and other deep learning algorithms shining in the research community. Reflecting on 2012, Lipson remarked that the field of artificial intelligence was like “the spring breeze of reform blowing everywhere.”
In 2012, Lipson’s team announced an open-source robot project called “Aracna Robot” based on their previous spider-like robot research, which research teams around the world can now use for their studies. Image source: QuantaIn early 2019, Lipson’s lab published a paper and video about a robotic arm that, through deep learning algorithms, built its virtual model from scratch using only its computational abilities—Lipson described this process as “similar to a non-verbal infant observing and recognizing its own hand for the first time.”
When this robotic arm was completely unaware of its position and shape, all it could do was perform random arm movements for an extended period to collect the necessary data. Image source:Columbia Engineering(engineering.columbia.edu/press-releases/lipson-self-aware-machines)After successfully building its virtual model, this robotic arm could accurately perform two different tasks: one, picking up a small ball from the table and placing it in a cup; and two, writing specific marks on paper—understanding, planning, and completing these tasks without any external assistance. Moreover, during this experiment, the researchers replaced a section of the robotic arm’s structure to simulate damage, and the robot not only detected this change but also updated its virtual model and still managed to complete the tasks correctly.
Image source:Columbia Engineering
Image source:Columbia EngineeringIndeed, such thinking is far from being well thought out, but Lipson firmly believes that this robotic arm has completed a qualitative transformation from reactive responses to deep thinking; the next step is merely a matter of quantitative change.“When you mention a robot’s self-awareness, people often think you mean the robot will suddenly wake up and say, ‘Hello? Who am I? Where am I?'” Lipson said, “But self-awareness does not exist in a black-and-white boundary; it can be as trivial as a thought like ‘Which direction should my hand reach out?’ In fact, this is the same kind of thinking as ‘Who am I? Where am I?’ but it focuses on more immediate questions.”This time, Quanta magazine interviewed Lipson, discussing how to define a robot’s self-awareness, what significance their consciousness holds, and what kind of future self-aware robots will lead humanity to. For clarity, we have edited the interview questions and answers.——————Q: You clearly have a great interest in the profound question of the essence of consciousness, but why do you insist on using robots to study consciousness? Why not choose the research directions of philosophers or neuroscientists?A: For me, the beauty of robotics is that it forces you to translate your subjective understanding into programs and algorithms, to immerse your thoughts into the world of mechanical principles. In the field of robotics research, you can never avoid the heavy lifting, nor can you use vague adjectives; you cannot say things like “life is a canvas,” as different people may have various interpretations of that. All these expressions are too ambiguous and cannot be translated to machines. Therefore, robotics forces you to be grounded.I also want to create something tangible, rather than just letting research remain at the level of words. Frankly speaking, philosophers have not made much progress in the past thousand years regarding the question of what consciousness is. This is not because there have not been outstanding philosophers during this millennium, nor is it because they lack interest in the question of consciousness—it is simply because you cannot study consciousness from a macro perspective.Of course, neuroscientists are approaching the essence of consciousness through more definitive and quantifiable research methods. However, I suspect they will eventually hit a bottleneck because they are still using a macro research perspective.Think about it: if you are trying to understand what consciousness is, why start with humans?Humans are the most complex conscious beings! It is like starting to climb a mountain from the steepest slope. We might as well change our perspective and find simpler systems, as they are likely easier to understand. That is what I am trying to do now: we are creating a robot with only four degrees of freedom (Degree of Freedom of Mechanism), which is obviously much smaller than human consciousness, but we can pose more specific questions, such as, “Can we make this machine create its own virtual model?”(Translator’s note: Degree of Freedom of Mechanism refers to the number of independent generalized coordinates required to determine the position of the mechanism.)
Image source: QuantaQ: Is the ability for self-awareness and self-simulation the same concept?A: One could say that a system capable of self-simulation has already achieved a certain degree of self-awareness. As for the extent to which it can self-simulate, whether in terms of the physical accuracy of its simulation or its awareness of itself being short-term or long-term—different experimental results can indicate the level of its self-awareness. This hypothesis is the basic starting point of our research.Q: In other words, you have concretized the abstract concept of “self-awareness” into “self-simulation,” which is a clearer standard in the technical field, representing a system’s ability to transform its spatial state into a virtual model?A: Exactly. I have proposed a unique definition for self-awareness and conducted research using this very precise definition. It can be calculated, measured, and quantified; you can even compute the error of this definition itself. Perhaps philosophers would say, “Uh, we wouldn’t understand self-awareness that way,” and then they would engage in some very vague discussions…Indeed, you can point out that our proposed definition is not true self-awareness, but you must admit that this definition is very practical and easy to study because we already have a standard (Benchmark). This standard is based on engineers writing lines of code to establish a robot’s virtual model. What we hope to see is not just that artificial intelligence algorithms learn to build their own virtual models, but that the AI models perform as well as, or even surpass, human-made models.
These artworks were created by a mass-produced artificial intelligence robot, with programs and algorithms specifically designed for painting. Image source: QuantaQ: Why is it necessary to create tangible robots? Can’t we study self-awareness in intangible virtual programs?A: I am a robotics scientist, so creating robots for research is naturally my first choice. In fact, what we expect to achieve is a closed system whose function is self-simulation. To enable this closed system to do so, we must provide it with certain inputs and observe its outputs—the key is that this research process must exist within certain boundaries; only within this closed environment can you possibly create a “self.” Robots are inherently suitable for such research; they can obtain inputs through perception, and they can output behavioral actions, all within a controllable range. They can encounter specific events and then simulate them.
This is another robot announced by Lipson’s team this year, called “Gray Goo.” This robot has light-sensitive devices and small motors but can only perform simple contraction movements. Image source:Columbia Engineering
When these robots form a swarm, they understand their relationships through algorithms, successfully modeling and finding an effective way to move as a group towards a light source. Image source:Columbia Engineering(engineering.columbia.edu/press-releases/hod-lipson-gray-goo)Q: Are these robots really creating their own models from scratch?A: Today, our research indeed allows robots to start from scratch to see how far they can go, and we consider this initial state a principle issue. However, in earlier research, such as with the spider-like robot, we did not have sufficient computational power. It was as if we had to tell it: “Listen, little robot, you currently have no idea where you are or where your parts are, but let me first tell you a physical law that I think is correct, F=ma, and now it’s your turn.”Q: How does artificial intelligence play a role in this process?A: I don’t know why, but humans are always keen to use artificial intelligence to help robots understand the external reality, while there is a strange enthusiasm for coding to understand what happens internally in the robot. Therefore, we focused on very trivial details from the beginning of our research, deciding that “humans have done a lot of software and hardware groundwork to help robots understand the external world, so we should leverage these to help robots learn what happens in their internal world, and this time the robots must rely on themselves.” If I had to summarize our research in one sentence, that is what we are doing.Q: The robot likely has to perform 1000 random actions to collect enough information for the deep learning algorithm to build its own model. Is this process what you mean by the robot being like a non-verbal infant?A: Exactly. When you see the robot flailing its arms randomly, it is likely trying to observe where the tip of its mechanical arm is. Imagine you activating your arm muscles to make a swinging motion, searching for the tip of your finger. For you, this is your input signal and output behavior. To accomplish this, the robot may have to flail around for over 30 hours, and only when we confirm it has collected enough data can we go home. From that moment on, it all depends on whether the deep learning algorithm can meet the challenge and create its own virtual model.Next, we increased the difficulty by disassembling the robotic arm and replacing a section with a specially deformed arm to simulate damage, and then we repeated the experiment. We witnessed the “injured” robot correcting the deformed part based on its overall model. This time, it no longer needed to start from scratch to create a model; although it still had to flail around for a while, the second data collection phase saved 90% of the time compared to the first experiment.
The red part in the image indicates the replaced mechanical arm structure, with changes in length and curvature. Image source:Columbia Engineering
Image source:Columbia EngineeringHowever, more importantly, before the robot begins its second flailing, it must first detect that something is wrong.Being able to do this is significant, but how can it detect the anomaly? When we humans experience a change in our bodies, we compare it with the virtual model in our minds; we can instantly know whether our hand is still in the original position just by looking. Or, if you expected to move 4 centimeters but suddenly find yourself 16 centimeters away, you can get feedback instantly. Similarly, the robot also detects anomalies in an instant. Then, it spends some time learning to adapt to these anomalies, and so on… But I must say, merely detecting anomalies is already a significant advancement.Q: Can this self-simulated model of the robot be likened to a certain brain region in humans? For example, a brain region that stores a diagram of body structure?A: That is how I think of it. Of course, this is also why the self-model created by the robotic arm looks rough and simplistic. After all, our little robot is just a mechanical arm that can flail around, with only four degrees of freedom. If we were to use a humanoid robot with over 800 degrees of freedom for this experiment, the artificial intelligence technology we have today would still be far from capable of handling such complex calculations.
Lipson’s team has also conducted another experiment, first allowing artificial intelligence to simulate 1000 square robots in a space, which could only move by flipping. However, after a while, the deep blue square robot on the right began to clear out an area that belonged only to it, while a cluster of cyan square robots began to form in the lower right corner. For some unknown reason, the artificial intelligence automatically rewarded itself through self-replication. Image source: TED
In the lab, Lipson’s team created a more powerful cube robot, and researchers found that as they continuously added small cube robots to the experimental environment, these robots would continuously connect with each other, building one pillar after another, also through this method of self-motivation. Image source: TED(www.ted.com/talks/hod_lipson_builds_self_aware_robots?language=en#t-8208)Q: If this is indeed a form of self-awareness, why give robots such capabilities? What significance does this have?A: Ultimately, it will make robots more plastic; indeed, you can design a robot by hand, as we typically do today. However, this is not only labor-intensive but also delays human time. When this robot undergoes deformation in the real world, such as being damaged or losing a wheel, or if one of its motors slows down, the model we originally designed becomes obsolete. Such problems are not easily solved because they are not merely about a screw being incorrectly installed on mass-produced robots.On the contrary, these issues are very serious; imagine a self-driving car. If you are willing to drive such a car, you are entrusting your life to an intelligent robot. You would want these robots to detect dangers and continuously monitor potential risks.Another reason is flexibility. Suppose a robot only performs one task; while doing that task, it continuously corrects and updates its model. If it suddenly needs to perform a new task, such as installing screws in another location, or if the new task is no longer screwing but spraying anti-rust coating, the robot can still use the same virtual model to continue learning how to complete the new task.Overall, this learning process is very similar to another algorithm in the field of artificial intelligence, called “zero-shot learning,” which refers to a type of deep learning that humans can perform—once you know how to climb a tree, even if faced with a strange tree in an unfamiliar environment, you can successfully climb it just by observing the trunk for a while.Similarly, once a robot has obtained its self-virtual model, it can engage in this level of learning: the only difference is that you won’t see the robot spending hours simulating the process of climbing the tree internally. From your perspective, you will only see a robot successfully completing a task, then pausing for a while, and immediately succeeding in completing another task without needing to practice or rehearse.Q: Currently, you are working on robots that can model themselves, and your goal is to create robots with self-awareness, or in simpler terms, robots with thinking abilities—how far are you from this grand goal?A: We have also conducted several other research projects focused on the self-modeling capabilities of robots, but this time not building their appearance but modeling their cognitive processes. We are making small steps in both directions, but ultimately these small steps will accumulate into a leap that will convince people that robots can not only achieve human cognitive levels but also surpass us.
Animation short film collection “Love, Death & Robots” features a segment called “Zima Blue.” Image source: TumblrQ: In other words, whether it is self-simulating the body or self-simulating the mind, you believe these two research directions will converge in the future?A: Yes, I believe these two types of research are essentially the same; this is our current hypothesis, and we are looking forward to the future of these studies.
***Book Giveaway***
“The Story of Our Ancestors”
An epic tale tracing the origins of life
Witness the extraordinary leap of human evolution over 4 billion years
The top 5 readers with the most likes on this article
(By noon tomorrow, limited to the WeChat public account platform)
Leave your contact information in the background
(Username + Address + Phone Number)
We will send a copy of “The Story of Our Ancestors” to 5 winners
If you miss out, you’ll have to pay for it yourself
Previous articles:
“Leviathan” (WeChat IDliweitan2014)
Weibo:Leviathan Planet
Neuroscience research, brain science, philosophy… a mix of everything. Anti-refreshing, anti-mind soup, anti-general pretentious literature, anti-fundamentals, anti-essence.
Submission email:[email protected]
Cooperation contact: WeChat ID liweitan2018
Click the mini-program or “Read the original text” to enter the store
☟