AI Empowerment in Combat Systems from an Agent Perspective

This article was published in “Command Control and Simulation” 2024, Issue 6

Liu Wei, Xie Haibin, Chen Shaofei. AI Empowerment in Combat Systems from an Agent Perspective[J]. Command Control and Simulation. 2024, 46(6): 8-14.

Abstract: This article addresses the issue of intelligent design in combat systems by proposing a conceptual framework and application methods for Agent-based artificial intelligence technology. First, the concept of Agent is clarified, and the significance of researching Agents in combat systems is discussed. Then, a research framework for Agent-based artificial intelligence is introduced, listing various application methods of Agents in combat systems. Finally, the development trends of Agent technology and the potential risks and challenges in combat applications are analyzed.

Combat systems refer to the totality of technical systems and organizational structures designed to achieve specific combat objectives, including functions such as intelligence collection, target identification, threat assessment, mission planning, and tactical execution. Modern warfare is characterized by complex combat elements, multi-domain cross-integration, and integrated joint operations, requiring troops to possess high adaptability and rapid response capabilities. Weapon systems should exhibit strong mobility and collaborative characteristics, and the overall combat system urgently needs intelligent upgrades. Artificial intelligence technology can enhance the situational awareness, command and control capabilities, decision-making efficiency, and precision and rapid response of military operations. However, how to conduct integrated intelligent design and upgrading of combat systems under a unified framework remains an important research topic.

This article proposes an Agent-based artificial intelligence technology framework and application methods for combat systems, exploring the application of Agents in combat systems by incorporating multi-domain and multi-dimensional combat space entities into a unified research framework, thus endowing combat systems with key characteristics such as autonomy, reactivity, and adaptability, enhancing the autonomous capabilities of combat systems, promoting multi-Agent collaboration, and improving adaptability in complex environments.

1 Concept and Connotation of Agent

1.1 Basic Concept of Agent

It is generally believed that an Agent is anything that can perceive the environment through sensors and influence the environment through actuators. The concept of Agent is highly inclusive, capable of encompassing all entities that perceive and affect the environment, whether living organisms or artificial objects. Combat systems involve diverse platforms and elements, such as human soldiers, unmanned combat equipment, intelligent software, etc., all of which can be included in the Agent category for design and analysis.

A multi-Agent system is composed of multiple Agents that interact to solve problems beyond individual capabilities or knowledge. Modern warfare involves multi-domain operations, requiring collaboration among multiple forces, coordination between manned and unmanned systems, and the integration of physical and virtual combat capabilities. Multi-Agent systems provide a conceptual framework for studying and solving distributed problems in combat operations, enabling effective organization, communication, and collaboration among heterogeneous entities based on Agent interaction.

1.2 Significance of Researching Agents

In combat systems, the significance of researching Agents mainly manifests in the following aspects:

1) By utilizing the autonomy and intelligence of Agents, the quality of combat decision-making and the speed of actions can be improved. Designing combat systems based on Agents can ensure good compatibility and scalability among various functional modules, enhancing intelligence analysis, situational awareness, planning, decision-making, and action execution capabilities in high-intensity confrontation environments.

2) The internal processing mechanisms of Agents are diverse, providing flexibility in technical choices for combat functional modules. Agents emphasize the external manifestation of intelligence and can select suitable artificial intelligence technologies based on the attributes of the task environment, not limited to human-like rational thinking or neural system information processing.

3) Through continuous learning and adaptation, Agents can better respond to complex battlefield environments. Agents continuously perceive the environment and exert influence on it, correcting their built-in environmental models based on interaction experiences, improving the mapping relationship between perception and action. With the ability to learn and evolve, Agents can break through their preset capability limits and adapt to dynamic and uncertain open environments.

2 Current Research Status of Agent-based Modeling Methods

Agent-based modeling is currently one of the most effective methods for simulating complex adaptive systems in combat. Combat Agents are mappings of various combat entities (which may differ in hierarchy and granularity) in simulation systems and are core elements of Agent-based modeling and simulation. Agent architecture represents the intermediate steps from specification to implementation, involving system structure selection, framework design, module decomposition and relationship determination, as well as modeling of perception and action mechanisms.

With the application of Agent-based modeling methods in practical modeling and simulation, software engineering concepts oriented towards Agents have emerged, proposing various methodologies such as knowledge engineering-based methods, object-oriented technology-based methods, and role and organization model-based methods. However, in practical applications, inconsistencies often exist between simulation models established by computer simulation engineers and military personnel’s military conceptual models, requiring repeated iterative corrections to meet military needs. Therefore, researchers have proposed a modeling method that directly drives multi-Agent combat simulation models with military conceptual models, improving engineering development efficiency while ensuring simulation credibility.

Researching complex systems like warfare for their adaptability to dynamic environments requires the use of multi-Agent modeling and simulation methods. Multi-Agent combat modeling and simulation utilize Agents to construct models of various entities in combat complex systems, describing the macro behavior of combat complex systems through the characterization of individual combat Agents and their interactions (including with the combat environment). Compared to traditional process-oriented and object-oriented simulation technologies, multi-Agent simulation methods possess stronger modeling and simulation capabilities for complex system behaviors, higher abstraction and expressive power, and greater simulation dynamics and flexibility, achieving an organic combination of micro and macro behaviors, making them an important means for studying complex combat systems.

In multi-Agent simulations, the adaptability of Agents to the combat simulation process and its results is emphasized, yielding fruitful research outcomes. For instance, Wang Buyun et al. summarized the achievements of genetic algorithms, reinforcement learning, neural networks, etc., in realizing the adaptability of combat Agents; Shi Ding et al. proposed reinforcement learning-driven multi-Agent collaborative combat simulation algorithms from the perspective of equilibrium decision-making. However, existing research often only realizes the adaptability of one side’s Agents, while the opposing side still adopts a fixed rule decision-making model. How to achieve simultaneous learning for both red and blue sides in simulations, whether the learning process can converge, and what impact it has on combat simulation outcomes are all issues that require in-depth research. As combat tasks become more complex and the composition of combat systems becomes more diverse, it is necessary to research and construct new combat systems that can cover various types of combat entities and adapt to complex and variable environments.

3 Research Framework for Intelligent Combat Systems Based on Agents

Based on the concept of Agents, a unified research framework can encompass the main branches of artificial intelligence (Figure 1). Individual Agent intelligence involves perceptual intelligence, cognitive intelligence, and action intelligence. Perceptual intelligence provides Agents with data acquisition capabilities and is the foundation of cognitive intelligence. Cognitive intelligence analyzes and makes decisions using perceptual data and is a prerequisite for action intelligence. Action intelligence implements decisions made by cognitive intelligence, translating thought into action. Collective intelligence is intelligent behavior exhibited through collaboration among multiple simple individuals, which can be combined with the aforementioned three types of intelligence to improve the overall effectiveness of the system. Hybrid intelligence integrates different types of intelligence to accomplish complex tasks and adapt to changing environments. This research framework is expected to provide a new paradigm for integrated intelligent design of combat systems.

Figure 1: Research Framework for Agent-based Artificial Intelligence

3.1 Perceptual Intelligence

Perceptual intelligence refers to the ability of machines to acquire information about their environment through sensors, establishing environmental models through information processing and understanding to express the information of their surroundings. Perceptual data processing is a key component of intelligence analysis and situational awareness. In image intelligence analysis, recognizing object targets, stitching terrain images, and detecting human targets are fundamental to high-level intelligence analysis such as target information acquisition, data fusion, target association, situational awareness, and target guidance. In electronic warfare and intelligence collection, voice recognition technology can automatically parse intercepted communication content, process variable signals and noise interference, adapt to different communication environments, and interpret communication content and intent, providing support for decision-making. It can also leverage text processing technology to produce accurate and standardized text processing products, providing technical support for intelligence analysis.

3.2 Cognitive Intelligence

Cognitive intelligence involves making judgments, reasoning, learning, planning, and forming decisions based on perceptual information and existing experience and knowledge, similar to human analysis, thinking, understanding, and judgment capabilities. In combat systems, the combination of knowledge bases and reasoning provides a powerful tool for analyzing complex data, improving decision quality, speed, and adaptability. Problem-solving can assist combat decision-making, battlefield simulation, strategic planning, and task allocation and path planning for unmanned systems. Intelligent planning and decision-making are core technologies for autonomous systems such as drones and unmanned vehicles, as well as software systems for simulation and combat mission planning. Machine learning enhances data analysis efficiency, threat assessment capabilities, battlefield situational awareness, and improves combat strategies, thereby increasing combat efficiency, warning capabilities, and decision accuracy.

3.3 Action Intelligence

Action intelligence is the ability of machines to translate decisions into actual actions, including intelligent control and robotic motion control. The realization of action intelligence relies on the comprehensive application of various control theory techniques, utilizing artificial intelligence technology to enhance autonomy and perform intelligent responses and operations in various environments. In combat systems, intelligent control uses advanced computational methods to optimize and automate the operation of weapon systems, enabling them to execute reconnaissance, surveillance, target location, and strike missions in complex environments. Motion control technology integrates perception, decision-making, and action, including individual motion control and group motion control, primarily applied to robots and unmanned systems.

3.4 Collective Intelligence

Collective intelligence is reflected in the intelligent behavior achieved through collaboration, aggregating numerous individual intelligences to realize complex intelligent actions. Multi-Agent systems help overcome the limitations of single Agents, enabling parallel processing of multiple processes and improving system operational reliability. In combat simulations, multi-Agent systems can be used to simulate interactions and collaborations among multiple forces, as well as applications in network-centric warfare, helping commanders understand and predict possible outcomes of actions. They can also transform military confrontations into game problems, utilizing artificial intelligence technology to solve issues such as intent judgment, threat assessment, and command and control, thereby enhancing decision-making levels.

3.5 Hybrid Intelligence

Hybrid intelligence aims for the deep integration of biological intelligence and machine intelligence, establishing a high-performance intelligent form that combines environmental perception, memory, reasoning, and learning capabilities of biology with the information integration, search, and computational capabilities of machines. In combat systems, the development of human-machine interaction technology enables humans to communicate and collaborate naturally and efficiently with machines, enhancing the efficiency and effectiveness of human-machine interaction. Currently, there are still significant challenges for unmanned systems to operate fully autonomously in complex dynamic environments, following the characteristics of “unmanned platforms, manned systems” to achieve human capabilities in environmental perception, situational judgment, task planning, decision-making, and action through intelligent interaction and collaborative control between humans and unmanned systems.

4 Application Methods of Agents in Combat Systems

Common application modes of Agents include: reflection, where Agents optimize decisions through interactive learning and reflection; tool usage, where Agents utilize various tools to complete tasks, which can be software applications, algorithms, hardware devices, or other resources that assist Agents in achieving their goals; planning, where Agents formulate a series of action steps to achieve objectives; and multi-Agent collaboration, which involves interaction and cooperation among multiple Agents. This article builds on these foundations and constructs single Agents based on reflection, combat mission planning Agents based on planning, parallel simulation Agents based on simulation, and multi-Agent systems based on collaboration, in conjunction with combat task requirements.

4.1 Concept of Digital Twin

On modern battlefields, the “fog of war” arises from the inability to quickly process and understand vast amounts of information. Agent technology can efficiently handle massive data sets to achieve precise situational awareness, providing a basis for decision-making and actions.

For example, in situational awareness, individual Agents contain basic modules such as configuration, memory, cognition, and action. The configuration module defines the basic characteristics and attributes of the Agent, such as identity, role, and capabilities. The memory module stores the Agent’s historical information and experiences, including past states, actions, and environmental feedback. In the cognitive module, the Agent forms comprehensive cognitive results based on its goals, current state, and environmental information. The action module provides combat suggestions and optimization strategies based on real-time situations and predictive analyses. Additionally, Agents may also include components for communication, multi-sensor fusion, and effect evaluation to enhance their effectiveness and adaptability in complex environments.

Reflection and introspection play a critical role in enhancing Agent performance. Agents reflect on past behaviors, learn from mistakes, and improve cognitive processes, which can enhance outcome quality. During reflection, the system evaluates the current behavioral output, compares it with expected results, and identifies problems or deficiencies. In the introspection process, based on the issues identified through reflection, it adjusts its situational cognitive processes or behavioral strategies, and develops new strategies or methods when necessary. By continuously self-evaluating and adjusting, the system can learn and improve cognitive processes, responding more effectively to complex problems.

4.2 Combat Mission Planning Agent and Parallel Simulation Agent Based on Planning

Applying Agents to combat mission planning and simulation allows for precise descriptions of combat entities in the battlefield environment, providing theoretical basis and strategic support for actual combat.Figure 2 shows the framework for Agent-based combat mission planning and parallel simulation.

Figure 2: Agent-based Combat Mission Planning and Parallel Simulation Framework

The combat mission planning Agent acquires multi-modal perception information through intelligent sensor networks, decomposes complex tasks based on given combat objectives in its internal planning system, formulates specific combat plans, and issues combat instructions to cluster combat units based on combat decisions. The intelligent sensor network consists of numerous sensor nodes, each capable of environmental information collection and data processing, actively perceiving the environment and communicating under scheduling. The combat mission planning Agent focuses on formulating plans and providing decision support, utilizing planning capabilities to determine the optimal course of action. Cluster combat units collaborate to complete tasks, providing action feedback to the planning system.

The parallel simulation Agent focuses on simulating different future scenarios to optimize decisions, testing the effectiveness of plans by creating possible future situations, helping decision-makers understand potential outcomes and risks. The parallel simulation Agent receives and processes real-time data, dynamically generating simulation entities and making adjustments based on real-time data to adapt to environmental changes. Through analysis and prediction of combat plans, it provides in-depth insights into future developments, supporting decision-making.

4.3 Collaborative Multi-Agent Systems

For complex tasks such as joint operations, intelligence analysis Agents, situational awareness Agents, combat mission planning Agents, parallel simulation Agents, and strategic support Agents form a multi-Agent system. Multi-Agent systems have the following advantages: 1) Numerical advantage, where each Agent engages in specific tasks, combining the skills and domain knowledge of multiple Agents to improve system efficiency and versatility; 2) Quality advantage, where multiple Agents may generate different viewpoints on the same issue, and through communication and combining their knowledge, Agents continuously update their viewpoints, effectively reducing false information and enhancing result reliability.

Interaction is key to multi-Agent collaborative work, with common interaction methods including cooperation, competition, negotiation, self-organization, and feedback. Different Agents can achieve effective division of labor and execution of complex tasks through role assignment and coordination. For example, CAMEL explored communicative Agents and proposed a social multi-Agent framework based on “role-playing”. By observing the behaviors and outcomes of other Agents, Agents learn new strategies and behavior patterns, improving their performance. Negative feedback can be used to suppress unwanted behaviors, promoting stable operation of the system. The system can also dynamically adjust the relationships and organizational structures among Agents based on task requirements and environmental changes.

In battlefield environments, communication may be disrupted, computing resources may be limited, and there may be risks of adversarial attacks, all of which impose high demands on multi-Agent system design. Researchers have developed various distributed decision-making algorithms, such as distributed Markov decision processes and multi-Agent reinforcement learning, to achieve collaborative combat in communication-constrained environments. Through self-organizing network technologies, multi-Agent systems can automatically adjust communication structures to maintain overall functionality even when communication is unstable or interrupted. Designing redundant systems and self-healing algorithms can enhance the system’s damage resistance and self-repair capabilities.

4.4 Examples of Agent Applications in Combat Systems

Assuming a complex battlefield terrain and variable weather, our combat mission is to seize the enemy’s command center in the city while protecting our critical infrastructure and minimizing the impact on civilians and urban facilities. We are equipped with the latest AI-driven combat systems, including unmanned combat aircraft, precision-guided weapons, and highly automated command and control systems. The enemy possesses a certain number of conventional armed forces, including infantry, tanks, and artillery, primarily relying on traditional tactics to execute defensive tasks.

The combat process includes five phases. In the initial phase, reconnaissance Agents deploy drones equipped with various sensors such as optical, infrared, and radar for all-weather reconnaissance, achieving real-time monitoring of enemy deployments through data fusion technology. Intelligence analysis Agents use deep learning algorithms to analyze the vast amount of data collected, quickly identifying vulnerabilities in enemy defenses and automatically generating attack suggestions. In the planning phase, the mission planning Agent evaluates the success rates of different attack paths and tactics using optimization algorithms based on the intelligence analysis results, automatically generating multiple combat plans for the commander to choose from. The combat simulation Agent, through high-precision battlefield simulation models, simulates the execution effects of different combat plans, predicting battlefield variables and potential risks. In the execution phase, the weapon control Agent directs drone swarms and precision-guided weapons to implement precise strikes while monitoring the effects of strikes and dynamically adjusting attack strategies. The combat execution Agent, supported by drones, swiftly mobilizes our elite forces to destroy key enemy targets using precise firepower. In the adjustment phase, the intelligence analysis Agent and mission planning Agent continuously monitor the battlefield situation, adjusting combat plans based on real-time intelligence, changing attack targets or reinforcements when necessary. In the concluding phase, all Agents collaborate to ensure complete control of the enemy’s command center while protecting our important facilities and civilian safety. The communication management Agent employs encryption communication technology to ensure secure communication and reliable information transmission during the combat process.

Within the combat system, a unified structural design is implemented for each Agent, with standardized input/output interfaces designed for each Agent to ensure they can receive and send information uniformly. Each Agent includes modules such as configuration, memory, cognition, and action, allowing for diversity in internal processing mechanisms based on different functionalities. A unified data format and protocol are established to ensure consistency in information exchange among different Agents, reducing parsing errors and delays. All Agents adhere to the same action protocols, specifying action strategies and response methods under specific circumstances, ensuring consistent and predictable behavior among Agents. Considering the high demands for reliability and real-time performance, a robust communication structure is designed to ensure that the system can continue to operate even when any Agent fails.

5 Trends in the Intelligent Development of Agent-based Combat Systems

With the advancement of large models and Agent technology, Agent systems are exhibiting trends of increased complexity, intelligence, and collaboration. Agents are expected to integrate the cognitive capabilities of large models, the action capabilities of embodied intelligence, and the collaborative capabilities of collective intelligence, playing a greater role in enhancing the performance of combat systems.

5.1 Enhancing Combat Agent Autonomy through Integration with Large Models

Large models have demonstrated tremendous potential in language understanding, decision-making, and generalization capabilities, presenting opportunities for Agents to break through. On one hand, the vast training sets of large models contain a wealth of human behavior data, laying the foundation for simulating human-like interactions; on the other hand, large models exhibit human-like thinking capabilities such as contextual learning, reasoning, and thinking chains, serving as the “brain” of Agents to decompose complex problems. Agents allow users to engage in complex interactions and task coordination, automatically creating appropriate prompts based on given objectives to better stimulate the reasoning capabilities of large models.

As shown inFigure 3, large model-driven Agents include modules for planning, memory, and tool invocation. The planning module is responsible for breaking down complex objectives into executable sub-tasks and dynamically adjusting execution strategies to flexibly respond to uncertainties. The task decomposition process is realized based on thinking chains, explicitly displaying the logical steps to solve problems, enhancing the transparency and interpretability of large models. The tool invocation module enables Agents to utilize external tools to perform specific tasks, such as database queries, scientific computing, and image processing. Tools serve as extensions of user capabilities, providing assistance in aspects such as specialization, factual accuracy, and interpretability. The memory module maintains context in multi-turn dialogues, preserving the world knowledge and existing experiences of the Agent.

Figure 3: Autonomous Agent System Framework Driven by Large Models

Combining large models with Agents can enhance the autonomy and decision-making speed of combat systems. Large language models demonstrate application potential in intelligence analysis, combat planning, and decision support. In April 2023, the U.S. Marine Corps tested the capabilities of the “Hermes” large language model in campaign-level combat planning. In May, the U.S. Army deployed the generative artificial intelligence system “Donovan” on a secure network for rapid intelligence analysis and decision support. In July, the U.S. Air Force tested five generative large models in the “Global Information Advantage” experiment to expedite the acquisition and processing of combat information. In July 2023, Yuanting Technology released the first domestic large model in the military field, “Tianji·Military,” integrating knowledge graphs, reinforcement learning, and operations optimization technologies to serve applications in combat command, unmanned swarm collaboration, and strategy simulation.

Large language model-based Agents can be used to simulate historical wars and future conflict scenarios, aiding in the analysis and prediction of various possibilities in warfare. For example, Hua et al. proposed the multi-Agent system WarAgent based on large language models, simulating several historical international conflicts. Although large models cannot fully replicate the complexity of human behavior, they show potential in dynamically simulating complex scenarios, dynamically hypothesizing scenarios, and calculating diplomatic policy decisions.

5.2 Autonomous Evolution of Combat Agents through Interactive Learning in Virtual and Physical Environments

In recent years, many new technologies under the concept of the metaverse, including augmented reality (AR), virtual reality (VR), head-mounted displays, 3D modeling, and artificial intelligence virtual environments, have been widely applied in the military field. Agents can serve as intelligent assistants in the metaverse, predicting, optimizing, and assessing military actions and weapon performance through simulation experiments, or as participants in virtual worlds, interacting and collaborating with humans.

Free exploration in virtual environments such as sandbox models provides sufficient interactive data and trial-and-error opportunities for Agent learning. Applications like AutoGPT, MetaGPT, CAMEL, and GPT Engineer exhibit remarkable diversity and powerful performance. By simply providing the role to play, description, and objectives, AutoGPT can autonomously decompose tasks, execute operations, and complete tasks using GPT-4 and search tools. In May 2023, NVIDIA’s AI Agent Voyager connected to GPT-4, autonomously writing code and dominating the game “Minecraft” with lifelong learning across the entire scene. Companies like SenseTime and Tsinghua University jointly proposed the versatile AI Agent GITM, capable of completing tasks through autonomous learning, demonstrating excellent performance.

Agents also demonstrate the ability to perform multi-modal understanding and learning in the physical world. Research on embodied intelligence focuses on the capability of Agents to operate effectively in the physical world and safely interact with humans in shared spaces, primarily through learning from demonstrations or feedback, enabling Agents to possess generalization capabilities in tool usage. Through continuous learning and reinforcement learning, Agents iterate and upgrade themselves through interactions with the environment, autonomously planning tasks, developing code, mobilizing tools, and optimizing paths to achieve objectives. In the future, Agents may autonomously manufacture tools, thereby enhancing their autonomy and independence.

5.3 Strict Risk Assessment Facing Practical Applications of Combat Agents

Applying combat Agents to autonomous decision-making and human-machine collaboration may raise safety, ethical, and legal issues, including how to ensure that Agent behaviors align with human values, how to prevent misuse or abuse, and how to prevent leaks and ensure data security. Risks associated with Agent applications include: 1) Ontological risks, where Agents may exceed human control; 2) Agency risks, where Agents may act without human directives; 3) Accidental risks, which may cause unintended harm due to system failures.

The maturity of Agent technology still needs improvement. Typically, Agents can only operate effectively within specific domains, and for roles that are difficult to characterize or transfer, targeted model fine-tuning is required to enhance performance. In the face of potential adversarial attacks, Agent designs need to be robust enough to prevent reliability issues caused by slight changes. Combat Agents need to possess strong adaptability and interpretability to undergo rigorous testing and regulation. Currently, there is a lack of unified standards for Agent performance evaluation, making it difficult to compare the strengths and weaknesses of different Agent systems. For the reasons mentioned above, most applications of Agents in combat systems will enhance rather than replace human roles.

6 Conclusion

As the complexity, adversarial nature, and real-time demands of warfare increase, it is anticipated that Agent technology will play a key role in optimizing combat systems. By integrating Agent technology into various aspects of combat systems, rapid target identification, threat assessment, and tactical execution can be achieved, making military operations more efficient and accurate. With the development of large language models, the ability of Agents to understand and generate natural language is continuously improving, leading to more natural and efficient human-machine collaboration. However, the application of Agents in combat systems still faces numerous difficulties and challenges. In resource-constrained open environments, the data analysis and reasoning capabilities of Agents are inadequate. It is necessary to introduce prior knowledge and utilize validation tools to compensate for the shortcomings of statistical data analysis. Additionally, coordination and regulation of Agent systems in technical, ethical, legal, and other aspects are required to ensure their safe, reliable, legal, and effective application in the military field.

END

| Authors: Liu Wei, Xie Haibin, Chen Shaofei

| Editor: Hu Qianjin

| Reviewer: Zhang Peipei

Corporate Mission: Enhance national defense with electronic equipment and serve society with technological industry

Corporate Vision: Build a leading innovative high-tech group in the field of information systems and intelligent equipment, both domestically and internationally

Core Values: People-oriented, Responsibility, Teamwork, Progress