This article was published in the fourth issue of “Medicine and Law” in 2018. Please note academic norms when citing.
Abstract: Artificial intelligence (AI) not only brings great convenience to humanity but also poses risks, such as criminal risks, where AI robots may become new subjects of crime that endanger human society. The industrial revolution has given rise to a risk society, which has necessitated the emergence of new criminal law theories, namely, risk criminal law theory. In the face of artificial intelligence, criminal law will inevitably undergo a new round of adjustments. In the era of weak artificial intelligence, the burden of proof in criminal cases will shift to the designers or operators of AI robots; in the era of strong artificial intelligence, AI robots will possess criminal responsibility capabilities, and we must consider the significance of punishment for both humans and AI robots, thereby formulating effective punitive measures against them.
1、Concept and Development Stages of Artificial Intelligence
Alan Mathison Turing (Alan Mathison Turing) was the first to propose the concept of artificial intelligence (AI), and he is also known as the father of artificial intelligence. Later generations have refined the concept of artificial intelligence. Ray Kurzweil explains that artificial intelligence is a discipline that involves delegating tasks that humans can perform to computers. [1] Robert Anderson believes that artificial intelligence is a branch of computer science that creates machines capable of exhibiting intelligent behavior. [2] It can be seen that there is still no unified definition of artificial intelligence in academia today. Wikipedia defines it as: machine intelligence, as opposed to natural intelligence, which is a computer program that makes rational actions based on environmental perception to maximize benefits. Thus, it can be seen that artificial intelligence is an intelligent computer program.
Artificial intelligence technology is a comprehensive technology built on three underlying technologies. First, image recognition technology, which refers to the technology of recognizing images to identify various different patterns and objects. The commonly used Google image search engine applies image recognition technology. Second, the foundational technology is speech recognition and natural language processing technology, such as binary coding or character sequences, which aims to convert the content of human speech into data that can be read by computers. Unlike the way humans recognize and confirm speech information, the latter attempts to identify or confirm the speaker of the speech information rather than the vocabulary contained within it. Successful technology companies in the field of speech recognition include iFlytek, Microsoft, etc. Natural language processing technology is applied in the field of machine translation, such as using machines to replace humans in simultaneous interpretation. The third foundational technology is knowledge graphs, which are a series of graphics that display the development process and structural relationships of knowledge, utilizing visualization technology to show knowledge resources and knowledge carriers, analyzing, mining, constructing, and displaying knowledge and the relationships between knowledge.
In today’s urban life, the application of artificial intelligence is ubiquitous. For example, when browsing products on Taobao or articles on Toutiao, the system automatically recommends related content based on our browsing history. Similarly, Google’s driverless technology, IBM Watson’s intelligent medical technology, and intelligent robots on automotive assembly lines. Big data has allowed the underlying technologies of artificial intelligence to leap from quantitative to qualitative changes. Before the application of big data, technologies such as image recognition and intelligent chatbots were still at a very primitive level. Thanks to the application of big data, artificial intelligence has surpassed humans in many deep fields, such as from AlphaGo to AlphaGo Zero, indicating that artificial intelligence has completely outperformed humans in the field of Go. However, in many broad horizontal fields, artificial intelligence is still in the stage of “artificial weak intelligence,” far from reaching the level of understanding the world and being adept in human emotions. This classification of the development stages of artificial intelligence is inherently consistent with the classification of the forms of artificial intelligence, which specifically manifests as: weak artificial intelligence (artificial narrow intelligence) and strong artificial intelligence (artificial general intelligence). Weak artificial intelligence can create robots that excel at reasoning and problem-solving, but they do not possess true intelligence. The fundamental difference between them and strong artificial intelligence lies in the presence or absence of consciousness. Strong artificial intelligence can match human capabilities in various aspects. Clearly, humanity is currently still in the era of weak artificial intelligence.
2、Risk Society and Criminal Risks of Artificial Intelligence
(1) Risk Society and Risk Criminal Law Theory
Industrial civilization has pushed humanity into a risk society, where the so-called risk society refers to a state in the post-industrial period where the risks and crises brought about by human practical activities pose serious threats to human production, life, and even survival, and humanity has lost control over these risks. The inherent flaws of industrial society have transformed it into a risk society, where risk has gradually become a fundamental characteristic of industrial society. [3] Risks in a risk society have the following characteristics: (1) Risks are human-made, with the widespread causes of risk and the broadening of affected subjects. (2) Risks have both positive and negative meanings. (3) The consequences of risks are extendable, manifesting in spatial expansion and temporal continuation. (4) The pathways of risk impact are uncertain, making recognition difficult.
Due to the inability of traditional criminal law’s principles of accountability to adapt to the risks brought about by risk society, new criminal law theories have emerged, namely, risk criminal law theory. The basic orientation of risk criminal law theory is that criminal law cannot ignore social life and indulge in closed logical theories; it must respond to social needs and be open to society. In a modern society where risks are ubiquitous, no matter how much people expect the protective function of criminal law, it cannot hinder the order-maintaining function of criminal law from becoming dominant. Therefore, criminal law has become an important tool for the state to combat risks, and criminal policy’s invasion of the public domain has become inevitable. As part of the social risk control mechanism, criminal law is no longer about punishment and retribution but about deterring social risks, with deterrence becoming the primary reason for imposing penalties. [4] To combat the criminal risks brought by industrial civilization, criminal law has gradually expanded its intervention scope, forming risk criminal law theory, which provides valuable references for criminal law to respond to the risks of the artificial intelligence era.
(2) The Era of Artificial Intelligence — An Upgraded Version of Risk Society
With the rapid development of artificial intelligence technology, humanity is increasingly concerned about the enormous risks it may bring. The late physicist Stephen Hawking warned: “In the next century, humanity will face the rise of AI robots that are smarter than themselves.” In 2015, Hawking again warned humanity at the Zeitgeist 2015 conference: “God has given humanity little time left; the rise of AI will mean the end of human history.” [5] Another notable figure warning about the risks of artificial intelligence is Elon Musk, the founder of Tesla, who believes that artificial intelligence will dominate humanity and become a permanent dictator. [6] Chinese scholar Professor Liu Xianquan believes that artificial intelligence technology will inevitably bring risks, including the potential amplification of certain traditional crimes, the emergence of new forms of crime, and even the possibility that AI robots may break free from human control and commit serious crimes that endanger society. [7] Clearly, intelligent machines capable of controlling or even destroying humanity will appear in the era of strong artificial intelligence, but in the current era of weak artificial intelligence, the criminal risks posed by artificial intelligence technology are also pressing issues that need to be addressed.
This article argues that the risks caused by artificial intelligence can be seen as an upgraded version of the risk society, and that criminal law should make adjustments similar to those made in response to the risk society when addressing the criminal risks of artificial intelligence. In the era of weak artificial intelligence, the level of intelligence of AI products is directly proportional to the level of human intervention; therefore, their designers and producers have the ability and obligation to control AI robots to prevent them from harming society. Thus, the author agrees with Professor Liu Xianquan’s proposal to establish a new crime of “abuse of artificial intelligence.” [7] At the same time, legislation should be forward-looking, and criminal law research should be no exception. According to Moore’s Law, in the foreseeable future, humanity will create robots with strong artificial intelligence; therefore, exploring the criminal responsibility of robots and the means of punishment is an inescapable responsibility for contemporary legal scholars.
3、Accountability of AI Robots in Criminal Law
(1) AI Robots in the Era of Weak Artificial Intelligence
As discussed earlier, AI robots in the weak artificial intelligence stage do not possess human attributes; they have certain capabilities similar to humans, but these capabilities are more akin to those of individuals with limited capacity for action, such as children or individuals with mental disorders. Therefore, AI robots in this stage have incomplete criminal responsibility capabilities and cannot independently bear criminal responsibility; any harm caused can only be attributed to the designers or operators behind the AI robots.
1. Criminal Responsibility of Designers or Operators
When the programmer or operator of an AI robot intentionally uses the AI robot to engage in socially harmful behavior, the AI robot is treated merely as a tool for committing a crime, with no legal distinction from ordinary animals or tools. For example, when a thief uses a screwdriver to open a window, he is using the screwdriver as a tool for committing a crime, and the screwdriver bears no criminal responsibility; the same applies to AI robots. For instance, if a programmer intentionally designs a program for an operational robot in a factory and places the robot inside the factory, and at night, the AI robot automatically starts its program and engages in criminal acts such as arson, the programmer should be regarded as a criminal. Similarly, if a person purchases a service-type AI robot and instructs it to attack an intruder in the yard, this is no different from commanding a pet dog to attack someone in real life. In this case, the operator of the AI robot should be recognized as the perpetrator of the crime committed by the AI robot.
2. Burden of Proof on Designers or Operators
The pathways of criminal risks posed by artificial intelligence are uncertain, and when it is impossible to confirm whether the designer or operator behind the AI robot has culpability, the burden of proof should fall on the designer or user of the program. For example, if a company produces an autonomous driving assistance system that malfunctions, leading to widespread traffic accidents involving vehicles equipped with that system, should this major accident be attributed to the company? In civil law, it is clear that the accident caused by the malfunction of the program should be attributed to the company. However, whether the company should be held criminally liable depends on whether it could foresee the serious consequences of the program malfunction at the time of product design and whether it took sufficient efforts to prevent such risks, such as conducting long-term large-scale testing and designing patches for potential program vulnerabilities. Clearly, a reasonable person would have the ability to foresee such risks, and if the company took sufficient efforts to avoid the risk, it may not be held criminally liable. The key issue is how to prove whether the company fulfilled its duty of care in the design of the AI product.
Under the traditional distribution of the burden of proof in criminal law, this burden should fall on the prosecution. However, in cases involving AI robots, the judiciary clearly lacks the capacity to bear this burden of proof. Fortunately, we can draw on the legislation of possession crimes, transferring the aforementioned burden of proof to the designers of AI robots. In China, Article 395 of the Criminal Law states that if a state employee’s expenditures or property significantly exceed their legal income and the discrepancy is substantial, they may be required to explain the source of the funds. If they cannot explain the legal source, the discrepancy will be deemed illegal income, and they may be sentenced to imprisonment for up to five years or criminal detention, and the discrepancy will be confiscated. This indicates that there are already provisions in Chinese criminal law for the transfer of the burden of proof; therefore, in the aforementioned autonomous driving accident case, we can legislate to assign the burden of proof to the company.
(2) AI Robots in the Era of Strong Artificial Intelligence
1. Criminal Intent of AI Robots
Regarding the theories of criminal composition, there are currently two main camps in China: the four-element theory and the three-tier theory. The three-tier theory holds that for a person to constitute a crime, they must meet three conditions: legality, unlawfulness, and culpability. The four-element theory divides the conditions for a person’s criminal composition into four elements: subject, subjective aspect, object, and objective aspect. Regardless of which theory is adopted, the essence of criminal composition can be distilled into subjective and objective aspects. In the case of AI robots, as long as the AI robot engages in behavior that harms society as defined by criminal law, it can satisfy the objective requirements for criminal composition. In contrast, determining the subjective aspect, especially the criminal intent of AI robots, is more complex.
In the era of strong artificial intelligence, AI systems possess good reception capabilities, and AI robots are equipped with sensory organs such as vision, sound, and touch. These receptors transfer the received data to the central processor for analysis, and this analytical process is similar to human understanding. By analyzing the data they receive, AI robots can have cognitive processes similar to those of humans. In fact, the so-called criminal intent refers to the purpose behind a person’s criminal behavior, and this purpose is not unique to humans; self-aware AI robots can certainly develop such purposes. Thus, it can be seen that AI robots in the era of strong artificial intelligence can generate criminal intent just like natural persons.
2. Criminal Responsibility of AI Robots
The bearer of criminal responsibility must possess the capacity for criminal responsibility, which means the ability to recognize and control one’s actions in the legal sense. [8] In the context of natural persons committing crimes, the ability to recognize refers to a person’s cognitive ability regarding the nature, significance, and consequences of their actions, while the ability to control refers to a person’s ability to act according to their will. [9] Children and individuals with mental disorders do not bear criminal responsibility due to their inability to recognize and control their actions; whether AI robots also have legal defenses against criminal responsibility is the first issue to consider when studying the criminal responsibility of AI robots.
As previously discussed, AI robots in the era of strong artificial intelligence possess self-awareness and even innovative thinking, and their capabilities can be compared to human intelligence. At this point, AI robots are no longer like children or individuals with mental disorders; when AI programs operate correctly, they will utilize their intelligent systems to analyze the data received by their receptors, thus possessing the recognition ability required for criminal responsibility. Additionally, AI programs can generally analyze the allowed and prohibited commands set by the system, so AI robots also possess the control ability required for criminal responsibility. Therefore, it can be seen that advanced algorithms running normally in AI robots generally do not have legal defenses against criminal responsibility.
3. Piercing the Veil of Artificial Intelligence
After granting AI robots the capacity for criminal responsibility, how can we prevent humans from using AI robots to commit socially harmful crimes? In the early days of human society, there was no concept of corporate crime, and even today, many countries still do not impose corporate criminal liability. China’s current criminal law combines general principles and specific provisions to regulate corporate crime. Granting corporate criminal liability has similarly brought about issues, namely, natural persons using corporations to commit crimes to evade legal sanctions. In response to this issue, legislators have provided corresponding solutions, namely, “piercing the corporate veil.”
The author believes that criminal legislation can draw on the recognition of corporate crime to “pierce the veil of AI robots.” In recognizing corporate crime, it is established that natural persons set up corporations to facilitate criminal activities or, after establishing a corporation, use it primarily for criminal purposes, which should not be recognized as corporate crime. [10] Similarly, traditional criminal subjects (natural persons and legal persons) designing AI robots for the purpose of committing illegal activities or traditional criminal subjects designing AI robots after their creation for the primary purpose of committing crimes should not only be held accountable for the criminal responsibility of AI robots but also have their criminal responsibility pursued.
4、Execution of Punishment for AI Robots
After AI robots are tried and convicted in the era of strong artificial intelligence, how should their punishments be executed? For example, if an AI robot is sentenced to one year in prison under specific circumstances, how should it serve its sentence? How can the death penalty be executed on AI robots? In cases where no individuals are arrested (especially those AI that are not installed on machines and have no physical form), what is the actual meaning of imprisonment? If an AI robot sentenced to a fine has no bank account or property of its own, what is the actual significance of the fine?
In light of these questions, we should first consider the following three issues: (1) What is the fundamental meaning of punishing natural persons? (2) How do traditional punitive measures affect AI robots? (3) What kind of punitive measures should be imposed on AI robots to achieve the purpose of punishment? Considering these three questions is the correct approach to addressing the issue of punishment for AI robots, and the greatest advantage of this approach is that it selects and adjusts punitive measures for AI robots based on commonly used punitive measures in modern society.
In summary, the death penalty is considered the most severe punishment for humans, and the “life” of AI robots exists as an independent entity. Sometimes it has a physical appearance (for example, as a robot); other times it only has an abstract existence (for example, as software installed on computer systems or network servers). Considering the effectiveness of the death penalty, imposing the death penalty on AI robots means deleting the control software of the AI robot. Once the deletion judgment is executed, the offending AI robot will no longer be able to commit any criminal acts, and deleting its software means eliminating the independent existence of the AI robot, which is effectively equivalent to executing the death penalty.
Imprisonment is one of the most common punitive measures against criminals in modern law. The significance of imprisonment for humans lies in depriving them of their rights to freedom and imposing strict supervision over their actions and life. The “freedom” of AI robots encompasses various freedoms, such as the freedom to participate in surgeries for medical service robots and the freedom to engage in production and manufacturing in factories. Given the significance of imprisonment for humans, when imposing a punishment on AI robots that has the same effect as imprisonment, the actual effect is to prevent the AI robot from applying its functions in specific areas for a certain period. In other words, during the period of imprisonment for AI robots, they are not allowed to act freely, nor are they allowed to perform their functions.
In most legal systems, community service is one of the punitive measures that can replace imprisonment. In China, community service is often combined with probation. As mentioned above, AI robots can participate in labor production in many fields. When AI robots work in factories, their purpose is to serve the interests of factory owners or other workers, thereby alleviating and facilitating human labor. Just as AI robots work for the interests of factories, they can also work for the public interest of the community, which essentially equates to sentencing AI robots to community service. Community service as a punitive measure is necessary for both humans and AI robots. Additionally, the purpose of probation is to prevent offenders from reoffending, and it does not require physical confinement; it only needs to be recorded in judicial records, which can also apply to AI robots.
[11] Monetary penalties have become an important punitive measure, and the significance of fines lies in depriving offenders of their property, which includes money and other assets. If an offender refuses to pay or is unable to pay the fine, the court often imposes other penalties to replace the fine. If AI robots in the era of strong artificial intelligence possess their own property, they can also be subject to fines. For natural persons and legal persons, the fines imposed may be their illegal gains or their legitimate labor income. When fines target the illegal labor income of offenders, their nature is equivalent to transferring property obtained through their own labor to the state. In other words, the amount of the fine can be transferred to the state in the form of labor. Therefore, when AI robots do not have sufficient property to pay fines, they can be compelled to provide labor for the public interest of society, effectively using equivalent labor to pay the fine. If the fine is paid in the form of labor for community benefit, it is similar to the community service sentence mentioned earlier.
5、Conclusion
Theories of criminal law and criminal legislation are products of the development of the times, used to address specific social issues of particular eras. Just as adjustments were made in criminal law theory and legislation in response to the emergence of a risk society, criminal law will inevitably undergo improvements in the era of artificial intelligence. The arrival of the era of weak artificial intelligence will change the traditional principles of accountability in criminal law, necessitating strict criminal responsibility for the designers or operators behind incidents caused by artificial intelligence, and shifting the burden of proof in criminal cases. In the era of strong artificial intelligence, criminal law theory and legislation should no longer limit the scope of criminal responsibility subjects to natural persons and legal persons; the view that “criminal law aims to regulate human behavior, and only legal norms based on scientific assumptions about human nature have rationality for existence and application” [12] should also be revised with the development of the times. The issue of executing penalties for AI robots that commit crimes in the era of strong artificial intelligence is a problem humanity must face. The author believes that we should first think about and identify the significance of punitive measures for AI robots and make appropriate adjustments based on existing punitive measures.
References
[1]Ray Kurzweil. The Singularity Is Near: When Humans Transcend Biology [J]. Viking. 2005, (1): 94-111
[2]Robert Anderson. The Impact of Information Technology on Judicial Administration: A Research Agenda for the Future [J]. 66s.CAL.L.REV, 1993, p.1807.
[3]Zhang Shaoqian. On the Two Basic Issues of Subjective Aspects of Major Environmental Pollution Accidents [J]. Zhongzhou Academic Journal, 2009, (3): 106-110.
[4]Yan Li. Inquiry into the Value Goals of Criminal Policy [J]. Political Law Forum, 2003, (5): 71-77.
[5]BBC News Stephen Hawking: will AI kill or save humankind? [E] [OL]. http://www.bbc.com/news/technology-37713629, accessed July 28, 2017.
[6]Andrew Arruda, An Ethical Obligation to User of Artificial Intelligence in Law and the Model Rules of Professional Responsibility [J]. American Journal of Trial Advocacy 443, Vol. 40, 2016, p.443.
[7]Liu Xianquan. Criminal Risks of Artificial Intelligence Era and Responses of Criminal Law [J]. Legal Business Research, 2018, (1): 3-11.
[8]Gao Mingxuan, Ma Kechang. Criminal Law [M]. Beijing: China University of Political Science and Law Press, 2016. 253.
[9]Qu Xinjiao. Criminal Law [M]. Beijing: China University of Political Science and Law Press, 2015. 191.
[10]Dong Yuting. On the Issue of Non-Unit Crimes Committed by Units [J]. Global Law Review, 2006, (6): 698-704.
[11]Hirano Shin, Robot Law — Towards Coexistence of AI and Humans [M]. Japan Kobundo, 2017. 248.
[12]Chen Xingliang. The Humanistic Foundation of Criminal Law [M]. Beijing: China Fangzheng Publishing House, 1999. 1.
Author’s Biography:
Li Xingchen, lawyer, holds a bachelor’s and master’s degree in law, with a solid foundation and expertise in the construction engineering field. He is currently a senior partner at Shanghai Hansheng Law Firm, providing legal services to several large and medium-sized construction contracting companies, successfully representing numerous significant and complex engineering dispute cases with a total controversy amount exceeding 1 billion yuan. He provides long-term legal advisory services to many enterprises, including Fortune 500 companies such as China Steel Group and several state-owned enterprises, central enterprises, and public institutions. Additionally, he has long provided specialized legal services for “Star Enterprises” in Yangpu District and several technology innovation enterprises.
1. Successfully represented a construction payment dispute case with nearly 200 million yuan in victory, with the actual constructor having priority compensation rights within 142 million yuan. 2. Successfully represented a civil retrial case for a Shanghai construction company, resulting in a successful appeal and retrial. 3. Successfully represented a civil retrial case for a construction company under China State Construction, resulting in a successful appeal and retrial (this case was rated as a “benchmark case” by the Law News Agency in 2024). 4. Represented a construction company in a construction contract dispute with Huashang Real Estate Development Company. 5. Represented a construction group company in a case where it was sued for construction delay losses, successfully dismissing all claims of over 60 million yuan. 6. Represented a construction company in a construction contract dispute, applying to the court to lift the preservation of nearly 80 million yuan without providing counter-guarantees or alternative property, reducing the sealed amount to 10 million yuan the following month, ensuring the normal operation of the enterprise. 7. Represented multiple disputes involving earthwork and civil engineering for a large general contracting unit. 8. Represented a Shanghai real estate company in a share transfer contract dispute with China Housing Exchange Construction Company.
Social Positions: External tutor for master’s students at Shanghai University of Political Science and Law, external tutor for master’s students at East China University of Science and Technology Law School, practical tutor for master’s students at East China University of Political Science and Law, part-time researcher at the Commercial Law Research Center of China University of Political Science and Law, part-time researcher at the Urban Safety Research Center of Shanghai University of Political Science and Law, legal review expert at the High-level Talent Center of Yangpu District, Shanghai, secretary of the Law Firm Norms and Development Committee of the Shanghai Bar Association, and investigator for the Disciplinary Committee of the Shanghai Bar Association.