AI Empowered Combat Systems Under Agent Perspective

This article was published in Command Control and Simulation, 2024, Issue 6.

Li Wei, Xie Haibin, Chen Shaofei. AI Empowered Combat Systems Under Agent Perspective[J]. Command Control and Simulation. 2024, 46(6): 8-14.

Abstract: This paper addresses the issue of intelligent design in combat systems by proposing a conceptual framework and application methods for AI technology based on agents. First, the concept of agents is elaborated, discussing the significance of researching agents in combat systems. Then, a research framework for AI based on agents is introduced, listing various application methods of agents in combat systems. Finally, the development trends of agent technology and the potential risks and challenges in combat applications are analyzed.

Combat systems refer to the totality of technical systems and organizational structures designed to achieve specific combat objectives, including functions such as intelligence collection, target identification, threat assessment, mission planning, and tactical execution. Modern warfare is characterized by the complexity of combat elements, multi-domain cross-integration, and integrated joint operations, which require forces to possess high adaptability and rapid response capabilities, while weaponry should exhibit strong mobility and collaboration characteristics. The overall combat system urgently needs intelligent upgrades. AI technology can enhance the situational awareness, command and control capabilities, decision-making efficiency, and precision and rapid response of military operations. However, how to conduct integrated intelligent design and upgrades of combat systems within a unified framework remains an important topic for research.

This paper proposes a framework and application methods for AI technology in combat systems based on agents, exploring the application methods of agents in combat systems by incorporating multi-domain and multi-dimensional combat space entities into a unified research framework, thereby endowing combat systems with key characteristics such as autonomy, responsiveness, and adaptability, enhancing the autonomous capabilities of combat systems, promoting multi-agent collaboration, and improving adaptability in complex environments.

1 The Concept and Connotation of Agents

1.1 Basic Concept of Agents

Generally, an agent is considered to be anything that can perceive its environment through sensors and affect the environment through actuators. The concept of agents is highly inclusive, capable of incorporating all individuals that perceive and influence the environment into a unified research framework, whether they are living beings or artificial entities. Combat systems involve diverse platforms and elements, such as human soldiers, unmanned combat equipment, intelligent software, etc., all of which can be included in the category of agents for design and analysis.

A multi-agent system consists of multiple agents that interact to solve problems beyond the capabilities or knowledge of individual agents. Modern warfare involves multi-domain operations, requiring coordination among multiple forces, collaboration between manned and unmanned systems, and integration of capabilities in both physical and virtual spaces. Multi-agent systems provide a conceptual framework for studying and solving distributed problems in combat operations, enabling effective organization, communication, and collaboration among distributed heterogeneous entities based on agent interactions.

1.2 The Significance of Studying Agents

In combat systems, the significance of studying agents is mainly reflected in the following aspects.

1) Utilizing the autonomy and intelligence of agents to improve the quality of combat decision-making and action speed. Designing combat systems based on agents can ensure good compatibility and scalability among various functional modules, enhancing capabilities in intelligence analysis, situational awareness, planning, decision-making, and action execution under high-intensity adversarial environments.

2) The internal processing mechanisms of agents are diverse, providing flexibility in technical choices for combat functional modules. Agents emphasize the external performance of intelligence, allowing for the selection of suitable AI technologies based on the attributes of the task environment, without being limited to human-like rational thinking or neural system information processing.

3) Through continuous learning and adaptation, agents can better cope with complex battlefield environments. Agents continuously perceive the environment and exert influence on it, correcting their built-in environmental models based on interaction experiences, thus improving the mapping relationship between perception and action. With learning and evolution capabilities, agents can break through their preset capability boundaries and adapt to dynamic and uncertain open environments.

2 Current Research Status of Agent-Based Modeling Methods

Agent-based modeling is currently one of the most effective methods for simulating complex adaptive systems in combat. Combat agents are mappings of various combat entities (which may differ in hierarchy and granularity) within simulation systems and are core elements of agent-based modeling and simulation. The agent architecture represents the intermediate steps from specification to implementation, involving multiple aspects such as system structure selection and framework design, module decomposition and relationship determination, and modeling of perception and action mechanisms.

With the application of agent-based modeling methods in practical modeling simulations, software engineering concepts oriented towards agents have emerged, proposing methodologies such as knowledge engineering-based methods, object-oriented technology-based methods, and role and organization model-based methods. However, in practical applications, inconsistencies often exist between simulation models established by computer simulation engineers and military personnel’s military conceptual models, requiring repeated iterative corrections to meet military needs. Therefore, researchers have proposed modeling methods that directly drive multi-agent combat simulation models using military conceptual models, improving engineering development efficiency while ensuring simulation credibility.

Researching complex systems like warfare for their adaptability to dynamic environments necessitates the use of multi-agent modeling and simulation methods. Multi-agent combat modeling and simulation utilize agents to construct models of various entities within complex combat systems, describing the macro behavior of combat complex systems through the characterization of individual combat agents and their interactions (including interactions with the combat environment). Compared to traditional process-oriented and object-oriented simulation technologies, multi-agent simulation methods possess stronger modeling and simulation capabilities for complex system behaviors, higher abstraction and expressiveness, and greater simulation dynamics and flexibility, achieving an organic combination of micro and macro behaviors, making it an important means for studying complex combat systems.

In multi-agent simulations, the adaptability of agents to combat simulation processes and their outcomes has been a focus of research, yielding fruitful results. For instance, Wang Buyun et al. have summarized the achievements of genetic algorithms, reinforcement learning, neural networks, and other methods in realizing combat agent adaptability; Shi Ding et al. proposed a multi-agent collaborative combat simulation algorithm driven by reinforcement learning from the perspective of balanced decision-making. However, existing research often only realizes the adaptability of one side of the agents, while the opposing side still adopts fixed rule decision-making models. How to achieve simultaneous learning for both red and blue sides in simulations, whether the learning process can converge, and its impact on combat simulation results are all issues that require in-depth research. As combat tasks become more complex and the composition of combat systems becomes more diverse, it is necessary to study the construction of new combat systems that can encompass various types of combat entities and adapt to complex and changing environments.

3 Intelligent Research Framework for Agent-Based Combat Systems

Based on the concept of agents, a unified research framework can encompass the main branches of artificial intelligence (Figure 1). Individual agent intelligence involves perceptual intelligence, cognitive intelligence, and action intelligence. Perceptual intelligence provides agents with data acquisition capabilities and forms the basis for cognitive intelligence. Cognitive intelligence analyzes and makes decisions based on perceptual data, serving as a prerequisite for action intelligence. Action intelligence implements decisions made by cognitive intelligence, transforming thought into action. Collective intelligence is the intelligent behavior exhibited through cooperation among multiple simple individuals, which can be combined with the aforementioned three types of intelligence to enhance the overall effectiveness of the system. Hybrid intelligence integrates different types of intelligence to accomplish complex tasks and adapt to changing environments. This research framework is expected to provide a new paradigm for integrated intelligent design of combat systems.

AI Empowered Combat Systems Under Agent Perspective

Figure 1: Research Framework for AI Based on Agents

3.1 Perceptual Intelligence

Perceptual intelligence refers to the capability of machines to acquire information about their environment through sensors, establishing environmental models through information processing and understanding to represent the information of their surroundings. Perceptual data processing is a key component of intelligence analysis and situational awareness. In image intelligence analysis, the recognition of object targets, stitching of terrain images, and detection of person targets are fundamental to high-level intelligence analysis such as target information acquisition, data fusion, target association, situational awareness, and target guidance. In electronic warfare and intelligence collection, voice recognition technology can automatically parse intercepted communication content, process variable signals and noise interference, adapt to different communication environments, and interpret communication content and intent, providing support for decision-making. It can also utilize text processing technology to produce accurate and standardized text processing products, offering technical support for intelligence analysis.

3.2 Cognitive Intelligence

Cognitive intelligence involves making judgments, reasoning, learning, planning, and forming decisions based on perceptual information and existing experience and knowledge, similar to human abilities such as analysis, thinking, understanding, and judgment. In combat systems, the combination of knowledge bases and reasoning provides powerful tools for analyzing complex data, improving decision-making quality, speed, and adaptability. Problem-solving can be used to assist combat decision-making, battlefield simulation, strategic planning, and task allocation and path planning for unmanned systems. Intelligent planning and decision-making are core technologies for autonomous systems such as drones and unmanned vehicles, as well as software systems for simulation and combat task planning. Machine learning enhances data analysis efficiency, strengthens threat assessment capabilities, optimizes battlefield situational awareness, and improves combat strategies, thereby increasing combat efficiency, warning capabilities, and decision-making accuracy.

3.3 Action Intelligence

Action intelligence is the capability of machines to translate decisions into actual actions, including intelligent control and robotic motion control. The realization of action intelligence relies on the comprehensive application of various control theory techniques, utilizing artificial intelligence technologies to enhance autonomy and perform intelligent responses and operations in various environments. In combat systems, intelligent control employs advanced computational methods to optimize and automate the operations of weapon systems, enabling them to perform reconnaissance, surveillance, target localization, and strike missions in complex environments. Motion control technology integrates perception, decision-making, and action, including individual motion control and group motion control, primarily applied in robotics and unmanned systems.

3.4 Collective Intelligence

Collective intelligence is manifested through cooperation among groups, aggregating numerous individual intelligences to achieve complex intelligent behaviors. Multi-agent systems help overcome the limitations of single agents, achieving parallel processing of multiple processes and enhancing the reliability of system operations. In combat simulations, multi-agent systems can be used to simulate interactions and collaborations among multiple forces, as well as applications in network-centric warfare, assisting commanders in understanding and predicting possible outcomes of actions. They can also transform military confrontations into game-theoretic problems, utilizing artificial intelligence technologies to solve issues such as intent judgment, threat assessment, and command and control, thereby enhancing decision-making levels.

3.5 Hybrid Intelligence

Hybrid intelligence aims for a deep integration of biological and machine intelligence, establishing a high-performance intelligent form that combines environmental perception, memory, reasoning, and learning capabilities of biological systems with information integration, search, and computational capabilities of machines. In combat systems, the development of human-machine interaction technologies enables humans to communicate and collaborate naturally and efficiently with machines, enhancing the efficiency and effectiveness of human-machine interaction. Currently, fully autonomous operation of unmanned systems in complex dynamic environments still faces significant challenges, adhering to the characteristic of “unmanned platforms, human systems” to achieve human capabilities in environmental perception, situational judgment, task planning, decision-making, and action through intelligent interaction and collaborative control between humans and unmanned systems.

4 Application Methods of Agents in Combat Systems

Common application modes of agents include: reflection, where agents optimize decisions through interactive learning and reflection; tool usage, where agents utilize various tools to accomplish tasks, which can be software applications, algorithms, hardware devices, or other resources that help agents achieve their objectives; planning, where agents devise a series of action steps to reach their goals; and multi-agent collaboration, involving interactions and cooperation among multiple agents. This paper builds on these foundations and combines them with combat mission requirements to construct single agents based on reflection, planning-based combat task planning agents, simulation-based parallel execution agents, and collaborative multi-agent systems.

4.1 The Concept of Digital Twins

On modern battlefields, the “fog of war” is caused by the inability to quickly process and understand a vast amount of information. Agent technology can efficiently handle massive data, achieving precise situational awareness to provide a basis for decision-making and action.

For instance, in situational awareness, a single agent contains basic modules such as configuration, memory, cognition, and action. The configuration module defines the basic characteristics and attributes of the agent, such as identity, role, and capabilities. The memory module is responsible for storing the agent’s historical information and experiences, including past states, actions, and environmental feedback. In the cognitive module, the agent forms comprehensive cognitive results based on its objectives, current state, and environmental information. The action module provides combat suggestions and optimization strategies based on real-time situations and predictive analysis. Furthermore, agents may also include components for communication, multi-sensor fusion, and effect assessment to enhance their effectiveness and adaptability in complex environments.

Reflection and introspection play a key role in enhancing agent performance. Agents reflect on past behaviors, learning from mistakes to improve cognitive processes, thereby enhancing outcome quality. During the reflection process, the system evaluates the current behavioral output, comparing it with expected results to identify problems or deficiencies. In the introspection process, based on identified issues from reflection, the agent adjusts its situational cognition process or behavioral strategy and develops new strategies or methods when necessary. Through continuous self-evaluation and adjustment, agents enable the system to learn and improve cognitive processes, effectively addressing complex problems.

4.2 Planning-Based Combat Task Planning Agents and Simulation-Based Parallel Execution Agents

Applying agents to combat task planning and simulation allows for precise descriptions of combat entities in battlefield environments, providing theoretical basis and strategic support for actual combat.Figure 2 illustrates the framework for agent-based combat task planning and parallel execution.

AI Empowered Combat Systems Under Agent Perspective

Figure 2: Framework for Agent-Based Combat Task Planning and Parallel Execution

The combat task planning agent acquires multi-modal perceptual information through intelligent sensor networks, decomposing complex tasks based on given combat objectives within its internal planning system, formulating specific combat plans, and issuing combat instructions to clustered combat units based on combat decisions. The intelligent sensor network consists of numerous sensor nodes, each capable of environmental information collection and data processing, actively perceiving the environment and communicating under scheduling. The combat task planning agent focuses on formulating plans and providing decision support, utilizing planning capabilities to determine the best action plans. Clustered combat units collaborate to accomplish tasks, providing action feedback to the planning system.

The parallel execution agent emphasizes simulating different future scenarios to optimize decisions, creating possible future situations to test the effectiveness of plans, helping decision-makers understand potential outcomes and risks. The parallel execution agent receives and processes real-time data, dynamically generating simulation entities and making adjustments based on real-time data to adapt to environmental changes. By analyzing and predicting combat plans, it provides in-depth insights into future developments, supporting decision-making.

4.3 Collaborative Multi-Agent Systems

For complex tasks such as joint operations, intelligence analysis agents, situational awareness agents, combat task planning agents, parallel execution agents, and strategic support agents form a multi-agent system. Multi-agent systems have the following advantages: 1) Quantity advantage, where each agent engages in specific tasks, combining the skills and domain knowledge of multiple agents to enhance system efficiency and versatility; 2) Quality advantage, where multiple agents may generate different viewpoints on the same issue, and agents continuously update their perspectives through communication and their own knowledge, effectively reducing misinformation and improving outcome reliability.

Interaction is key to multi-agent collaborative work, with common interaction modes including cooperation, competition, negotiation, self-organization, and feedback. Different agents can achieve effective division of labor and execution of complex tasks through role assignment and coordination. For example, CAMEL explored communicative agents and proposed a social multi-agent framework of “role-playing.” By observing the behaviors and outcomes of other agents, agents learn new strategies and behavioral patterns to enhance their performance. Negative feedback can be used to suppress undesirable behaviors, promoting stable system operation. The system can also dynamically adjust the relationships and organizational structures among agents based on task requirements and environmental changes.

In battlefield environments, communication may be disrupted, computational resources may be limited, and there may be risks of adversarial attacks, all of which impose high demands on multi-agent system design. Researchers have developed various distributed decision-making algorithms, such as distributed Markov decision processes and multi-agent reinforcement learning, to achieve collaborative combat in communication-constrained environments. Through self-organizing network technologies, multi-agent systems can automatically adjust communication structures during unstable or interrupted communications, maintaining overall system functionality. Designing redundant systems and self-healing algorithms can enhance the system’s damage resistance and self-repair capabilities.

4.4 Examples of Agent Applications in Combat Systems

Assuming a complex battlefield terrain and changing weather, our combat mission is to seize the enemy’s command center in the city center while protecting our critical infrastructure and minimizing impacts on civilians and city facilities. We are equipped with the latest AI-driven combat systems, including unmanned combat aircraft, precision-guided weapons, and highly automated command and control systems. The enemy possesses a certain number of traditional armed forces, including infantry, tanks, and artillery, primarily relying on traditional tactics to execute defensive tasks.

The combat process includes five stages. In the initial stage, reconnaissance agents deploy drones equipped with various sensors such as optical, infrared, and radar for all-weather reconnaissance, achieving real-time monitoring of enemy deployments through data fusion technologies. Intelligence analysis agents utilize deep learning algorithms to analyze the vast amounts of data collected, quickly identifying vulnerabilities in enemy defenses and automatically generating attack suggestions. During the planning stage, the task planning agent assesses the success rates of different attack paths and tactics based on intelligence analysis results, automatically generating multiple combat plans for commanders to choose from. The combat simulation agent uses high-precision battlefield simulation models to simulate the execution effects of different combat plans, predicting battlefield variables and potential risks. In the execution stage, the weapon control agent directs drone swarms and precision-guided weapons to implement precise strikes while monitoring strike effects and dynamically adjusting attack strategies. The combat execution agent, supported by drones, rapidly maneuvers our elite forces to destroy key enemy targets using precise firepower. In the adjustment stage, intelligence analysis agents and task planning agents continuously monitor battlefield situations, adjusting combat plans based on real-time intelligence, changing attack targets or providing reinforcements when necessary. In the concluding stage, all agents collaborate to ensure complete control of the enemy’s command center while protecting our important facilities and civilian safety. The communication management agent employs encrypted communication technologies to ensure secure communication and reliable information transmission during the combat process.

Within the combat system, a unified structural design is implemented for each agent, with standardized input/output interfaces designed for each agent to ensure they can receive and send information in a uniform manner. Each agent includes modules such as configuration, memory, cognition, and action, allowing for diversity in internal processing mechanisms based on different functions. A unified data format and protocol are established to ensure consistency in information exchange among different agents, reducing parsing errors and delays. All agents follow the same set of action protocols, stipulating action strategies and response methods in specific situations, thereby ensuring consistent and predictable behavior among agents. Given the high requirements for reliability and real-time performance, a robust communication structure is designed to ensure that the system continues to operate even if any agent fails.

5 Trends in the Intelligent Development of Agent-Based Combat Systems

With the advancement of large models and agent technologies, agent systems are exhibiting increasingly complex, intelligent, and collaborative development trends. Agents are expected to integrate the cognitive capabilities of large models, the action capabilities of embodied intelligence, and the collaborative capabilities of collective intelligence, playing a greater role in enhancing the effectiveness of combat systems.

5.1 Enhancing Combat Agent Autonomy through Large Models

Large models have demonstrated immense potential in language understanding, decision-making, and generalization capabilities, providing opportunities for agents to break through existing limitations. On one hand, the vast training sets of large models contain extensive human behavior data, laying the groundwork for simulating human-like interactions; on the other hand, large models exhibit human-like thinking abilities such as context learning, reasoning, and thinking chains, serving as the “brain” of agents to decompose complex problems. Agents allow users to engage in complex interactions and task coordination, automatically creating appropriate prompts based on given objectives, better stimulating the reasoning capabilities of large models.

As illustrated in Figure 3, large model-driven agents include modules for planning, memory, and tool invocation. The planning module is responsible for decomposing complex objectives into executable subtasks and dynamically adjusting execution strategies to flexibly respond to uncertainties. The task decomposition process is realized based on thinking chains, enhancing the transparency and interpretability of large models by clearly displaying the logical steps for problem-solving. The tool invocation module enables agents to utilize external tools to perform specific tasks, such as database queries, scientific calculations, and image processing. Tools, as extensions of user capabilities, can provide support in terms of specialization, factuality, and interpretability. The memory module maintains context across multiple rounds of dialogue, preserving the agent’s world knowledge and existing experiences.

AI Empowered Combat Systems Under Agent Perspective

Figure 3: Autonomous Agent System Framework Driven by Large Models

Combining large models with agents can enhance the autonomy and decision-making speed of combat systems. Large language models demonstrate application potential in intelligence analysis, combat planning, and decision support. In April 2023, the U.S. Marine Corps tested the capabilities of the “Hermes” large language model in campaign-level combat planning. In May, the U.S. Army deployed the generative AI system “Donovan” on a secure network for rapid intelligence analysis and decision support. In July, the U.S. Air Force tested five generative large models in the “Global Information Advantage” experiment to improve the speed of acquiring and processing combat information. In July 2023, YuanTing Technology released the first domestic large model in the military field, “Tianji·Military,” integrating knowledge graphs, reinforcement learning, and operational optimization technologies, serving applications in combat command, unmanned swarm collaboration, and strategy simulation.

Large language model-based agents can be used to simulate historical wars and future conflict scenarios, aiding in the analysis and prediction of various possibilities of warfare. For instance, Hua et al. proposed the multi-agent system WarAgent based on large language models, simulating multiple historical international conflicts. Although large models cannot fully replicate the complexity of human behavior, they exhibit potential in reconstructing complex situations, dynamically hypothesizing scenarios, and calculating decisions in foreign policy.

5.2 Autonomous Evolution of Combat Agents through Interactive Learning in Virtual and Physical Environments

In recent years, many new technologies under the metaverse concept, including Augmented Reality (AR), Virtual Reality (VR), head-mounted displays, 3D modeling, and AI virtual environments, have been widely applied in the military field. Agents can serve as intelligent assistants in the metaverse, predicting, optimizing, and assessing military actions and weapon performance through simulated experiments or participating as entities in virtual worlds, interacting and collaborating with humans.

Free exploration in virtual environments such as sandbox models provides agents with sufficient interactive data and trial-and-error opportunities for learning. Applications like AutoGPT, MetaGPT, CAMEL, and GPT Engineer have demonstrated remarkable diversity and powerful performance. By simply providing a role to play, descriptions, and objectives, AutoGPT can autonomously decompose tasks, execute operations, and complete objectives using tools like GPT-4 and search. In May 2023, NVIDIA’s AI Agent Voyager integrated with GPT-4, autonomously writing code and dominating “Minecraft,” engaging in lifelong learning across the entire game scene. Companies like SenseTime and Tsinghua University jointly proposed the generalist AI Agent GITM, capable of autonomously completing tasks with outstanding performance.

Agents have also demonstrated the ability to perform multi-modal understanding and learning in the physical world. Research on embodied intelligence focuses on the ability of agents to operate effectively in the physical world and safely interact with humans in shared spaces, primarily learning from demonstrations or feedback, enabling agents to develop generalization capabilities in tool usage. Through continuous learning and reinforcement learning, agents iteratively upgrade themselves through interaction with the environment, autonomously planning tasks, developing code, mobilizing tools, and optimizing paths to achieve objectives. In the future, agents may autonomously manufacture tools, thereby enhancing their autonomy and independence.

5.3 Strict Risk Assessments for Practical Applications of Combat Agents

The application of combat agents in autonomous decision-making and human-machine collaboration may raise security, ethical, and legal issues, including how to ensure that agent behaviors align with human values, how to prevent misuse or abuse, and how to prevent leaks and ensure data security. Risks associated with agent applications include: 1) Ontological risks, where agents may exceed human control; 2) Agency risks, where agents may take actions without human directives; 3) Accidental risks, which may arise from system failures leading to unintended harm.

The maturity of agent technology still requires improvement. Typically, agents can only operate effectively in specific domains, and for roles that are difficult to represent or transfer, targeted model fine-tuning is necessary to enhance performance. Given potential adversarial attacks, agent designs need to be sufficiently robust to prevent reliability issues caused by minor changes. Combat agents need to possess strong adaptability and interpretability to undergo rigorous testing and regulation. Currently, there is a lack of unified standards for evaluating agent performance, making it challenging to compare different agent systems. For these reasons, most applications of agents in combat systems will enhance rather than replace human roles.

6 Conclusion

As the complexity, adversarial nature, and real-time requirements of warfare increase, it is expected that agent technology will play a key role in optimizing combat systems. By integrating agent technology into various aspects of combat systems, rapid target recognition, threat assessment, and tactical execution can be achieved, making military operations more efficient and accurate. With the development of large language models, agents’ capabilities in understanding and generating natural language are continually improving, leading to more natural and efficient human-machine collaboration. However, the application of agents in combat systems still faces numerous difficulties and challenges. In resource-constrained open environments, agents’ data analysis and reasoning may be insufficient. It is necessary to introduce prior knowledge and use verification tools to compensate for the shortcomings of statistical data analysis. Additionally, coordination and control of agent systems in technical, ethical, legal, and other aspects are required to ensure their safe, reliable, legal, and effective application in the military field.

END

| Authors: Li Wei, Xie Haibin, Chen Shaofei

| Editor: Hu Qianjin

| Reviewer: Zhang Peipei

Source: Command Control and Simulation

Original link: https://mp.weixin.qq.com/s/VW7QmrkEVMNUSrwPBu7Cig