1 Introduction
The Art Performance System (APS) is a comprehensive, open, integrated creative platform serving the performance axis, which includes subsystems for light art presentation, sound art presentation, visual art presentation, and form art presentation. It has technical characteristics such as distribution, autonomy, dynamism, and complexity. The integration mentioned here refers to: first, the fusion of the real-time operating environments of various subsystems related to performance presentation, that is, collecting the operating environment information of each subsystem for centralized management, thus forming the overall real-time operating environment of the performance presentation system; second, the integrated control of the operation and maintenance of performance equipment managed by each subsystem, which enhances the onsite audio-visual and artistic creative work presentation and experience; third, the centralized management, maintenance, and presentation of performance information from various performance presentation subsystems, including shows, sequences, cues, etc., in a centralized control or coordinated autonomous control manner, allowing for flexible processing and centralized operational supervision. Therefore, the performance presentation system is a dynamic, complex system distributed in space and time, and it is an important component of performance venue management.
Practical experience has shown that traditional regulatory methods and technologies are quite ineffective in the complex performance field. The characteristic of the performing arts industry is the autonomy of various subsystems, which often requires human intervention in an important performance to ensure the safe operation of the performance. Intelligent systems, due to the introduction of artificial intelligence technologies such as expert systems, fuzzy logic, artificial neural networks, and genetic algorithms, do not solely rely on models for the realization of adaptive, self-organizing, and self-learning functions, but place greater importance on the core role of knowledge. By transforming the management control experience of experts into usable late-developing knowledge or using machine learning methods to achieve automatic knowledge acquisition, they demonstrate advantages in the field of complex systems.
Multi-Agent, as a new technology in Distributed Artificial Intelligence (DAI), has become a research hotspot and has achieved significant application results. Multi-Agent Systems (MAS) are mainly used for the coordination of intelligent behavior among a group of autonomous intelligent agents, coordinating their knowledge, goals, skills, and planning to take joint actions or solve problems. MAS has been successfully applied in complex systems such as autonomous driving, advanced robotic systems, information retrieval, and highways. The distribution, dynamism, complexity, and autonomy of the various subsystems in the performance presentation system align with the technical characteristics of MAS. It is entirely feasible to apply MAS technology to solve the complex integration problems in performance presentation, which should attract industry attention. Based on the analysis of the hierarchical structure of the technical equipment system in the performance environment, the author proposes a distributed hierarchical system architecture for the performance presentation system. By introducing intelligent agents, an intelligent performance presentation system MAS model is established, and the structural model, construction methods, communication mechanisms, and negotiation mechanisms of the agents in this MAS are analyzed.
2 Distributed Hierarchical System Architecture of the Performance Presentation System
The technical equipment in the performance environment is a typical distributed hierarchical structure from a logical perspective, as shown in Figure 1. The regulatory logic of the distributed hierarchical system adopts the principles of grouping and coordination. Through horizontal (spatial) grouping and vertical functional decomposition, the entire system can be decomposed into several subsystems that complete independent artistic presentation processing, while the regulatory control function is decomposed to different levels, supplemented by local regulation, processing, or control methods to achieve complex artistic presentation tasks and goals. This structure can relatively effectively address the safety and reliability issues of complex systems. Therefore, the architecture of the performance presentation system adopts a distributed hierarchical approach, dividing it into a presentation layer, control layer, and equipment layer, characterized by integration, safety, and convenience, to meet the application requirements of the performing arts industry and enhance the onsite experience of artistic presentation and creative works centered around performances. From a logical perspective, the distributed hierarchical system includes the following levels.
-
Presentation Layer: The core of the performance presentation system platform manages and maintains the state parameters and various performance information of each subsystem, presenting the effects of grouping performance scenes. The performance presentation system platform communicates with the control processing units of each subsystem in the control layer via Ethernet (ETHERNET), coordinating and scheduling uniformly. In practical applications, the performance information files of each presentation subsystem, such as shows, sequences, and scenes, will present different management and control methods during rehearsals and performances. Some shows operate according to a timeline, which generally adopts the automatic time code (Time Code) control method (also known as “Show Control”), allowing the performance presentation system platform to organize the operation of performance files centrally. Some important performances often require human intervention from various subsystems to ensure the safe operation of the performance; this method will adopt an autonomous control mode for each presentation subsystem, with the performance presentation system platform coordinating the operation of performance files.
-
Control Layer:Consists of control processing units from various presentation subsystems, which execute the regulatory functions of the equipment operations of each presentation subsystem and report real-time operating environment, equipment operation maintenance management information, and performance information to the performance presentation system platform in the presentation layer, communicating via Ethernet with the performance presentation system platform. The physical locations of the control processing units in each subsystem may vary; for example, in the sound presentation system, the control units may be distributed among digital mixing consoles or DSP processing equipment, or they may be a separate management computer. The control units in the light presentation subsystem are generally found in lighting control desks; the control processing units in the visual presentation subsystem are typically located within media servers or video production processing cores; and the control processing units for stage machinery are usually found in stage machinery control consoles. Other control processing units of presentation subsystems may be located in central control, management computers, safety servers, or monitoring dispatch machines.
-
Equipment Layer:Consists of controlled devices related to each subsystem, which generally include adjustment, processing, control, management devices, data transmission and exchange devices, presentation terminal devices, etc., and vary according to the different presentation subsystems. For example, the sound presentation system may include onsite pickup devices (MIC), while the visual presentation subsystem may have audio and video recording and playback devices. The primary task of the equipment layer is to present and enhance the onsite experience of the performance content. The performance presentation system, utilizing a distributed hierarchical logical structure, possesses technical characteristics of distribution, autonomy, dynamism, and complexity, and combined with digital and network integration applications, meets the performing arts industry’s requirements for “convenience, reliability, safety, diversity, innovation, open extensibility, compatibility, and flexibility.”
3 Introduction to Intelligent Agents
Since the 1980s, intelligent agents have been a research hotspot in distributed artificial intelligence. Although the term has been widely used, there is still no universally accepted definition. Agents are typically defined as a type of “computer system based on a certain scenario, with flexible and autonomous behavior capabilities to meet design goals,” generally referring to hardware or software-based computer systems that meet the following characteristics.
-
Autonomy: Agents operate without direct intervention from humans or other agents and can exercise some control over their own behavior and internal states.
-
Social Ability: They can interact with other agents (which may also include humans) through some agent communication language. There are three main types of interaction: cooperation, coordination, and negotiation.
-
Reactivity: They can perceive their environment (which can be the physical world, a user connected via a specific user interface, a series of other agents, the Internet, or a combination of all these) and respond promptly to changes in the environment.
-
Proactivity: Agents not only respond to the environment but can also take proactive actions to achieve their goals.
Agents are also defined as computer systems that, in addition to meeting all the above characteristics, possess concepts typically applied to humans, such as knowledge, belief, intention, obligation, and even emotional factors. Additionally, many scholars have proposed that agents have characteristics such as situationality, flexibility, mobility, honesty, friendliness, rationality, longevity, and adaptability.
Multi-Agent Systems (MAS) refer to cooperative systems composed of multiple autonomous intelligent agents. It is a loosely coupled problem-solving network, with the goal of solving problems that exceed the individual capabilities or knowledge of each problem solver. These problem solvers are agents that are autonomous and may be heterogeneous. To deeply understand the concepts of agents and multi-agent systems, it is essential to distinguish them from concepts such as objects, process control systems, software “sprites,” and distributed problem solving. The focus of multi-agent systems (MAS) research is on coordinating intelligent behavior among a group of autonomous intelligent agents, coordinating their knowledge, goals, skills, and planning to take joint actions or solve problems.
The performing arts industry is characterized by the distribution and autonomy of various subsystems. An important performance often requires human intervention from multiple presentation subsystems to ensure the safe operation of the performance. Practical experience indicates that complex systems like performance presentation require exploring new intelligent methods to solve issues related to safety integration, integrated presentation, and real-time supervision. MAS, by introducing artificial intelligence technologies, can adapt well to the distributed and autonomous technical requirements of performance presentation systems, thus demonstrating advantages in the field of complex systems for performance presentation.
4 MAS Modeling of the Performance Presentation System
The performance presentation system adopts a distributed hierarchical structure to complete the following three main functions:
-
Using an open standard secure platform, integrating performance-related subsystems, including the real-time operating environment parameters, equipment operation maintenance management information, and performance information of each presentation subsystem;
-
Introducing intelligent technology to present the connotation of “Internet+” service, achieving flexible and convenient management of intelligent performance file operations, enhancing onsite regulatory effects;
-
Leveraging networked and digital means to integrate heterogeneous protocol communication interfaces and various control methods, optimizing the integration of technical equipment in the performance environment, better serving performance realization and safety. To meet the functional requirements of integration, safety, and convenience in the performance presentation system, it is necessary to establish a multi-agent model for the performance presentation system. The primary task is to convert each functional module of the performance presentation system into autonomous intelligent agents with independent functions and establish the functional structures of each agent based on their respective functions. Based on multi-agent theory and the analysis of the distributed hierarchical system structure of the performance presentation system, the author proposes the structure of the intelligent performance presentation system based on Multi-Agent, as shown in Figure 2.
In the intelligent performance presentation system shown in Figure 2, multiple agents form an organized and orderly group that works together in the “environment.” Their organization and orderliness are reflected in the three different agents in the presentation layer (Organizational Agent, Coordination Agent, and Safety Agent) managing several control agents in the control layer; each control agent manages the performance presentation objects and operating environments of various presentation subsystems. This organizational structure is similar to human group organization, where leaders and subordinates collaborate to complete specific tasks. The Organizational Agent operates in performance scenarios that do not require human intervention, such as completing the running of various performance files according to time code (Time Code) in show performances and coordinating the automatic operation of various presentation subsystems; the Coordination Agent operates in important performances that require human intervention, coordinating the autonomous operation modes of various presentation subsystems; the Safety Agent is focused on safety management and coordination within the integrated environment of the entire performance system and maintains a close relationship with the Organizational Agent and Coordination Agent in the presentation layer for comprehensive safety management and coordination. Each control agent selectively executes the control commands from the agents in the presentation layer based on the current operating states and environmental information of each subsystem, where the commands from the Organizational Agent and Coordination Agent are mutually exclusive for the control agents, meaning that a control agent cannot execute commands from both agents simultaneously. Each agent operates in the environment, completing their respective tasks based on environmental information. At the same time, MAS also interacts with the environment, such as when the control system interacts with humans through an intelligent human-machine interface. Moreover, this MAS design retains interfaces for future intelligent upgrades of performance venues, allowing agents to interact with other similar agent systems (such as stage management MAS) to form a higher-level organized MAS. In the MAS of the performance presentation system, agents adopt a composite structure, as shown in Figure 3.
In Figure 3, the “environment” is broadly defined, which can be the external world, controlled objects, or even humans. Perceptors receive environmental information through protocols such as RDM, performing preprocessing and feature identification. If a simple or urgent situation is perceived, the information is sent to the reflector. The reflector immediately decides on the incoming information and sends action commands to the executor, which takes corresponding actions. If a complex situation is perceived that allows for relatively ample time for processing, the information is sent to the modeling module for analysis. The modeling module includes models established by agents regarding the environment and other agents, which can predict short-term situations based on the model and current perceptions, thus proposing corresponding countermeasures to the decision-making module. At the same time, other agents can request their future actions through the communication module, and the planner makes plans for their actions, with the decision-making module determining the corresponding actions to be taken.
5 Implementation Technologies of Agents in MAS
In the MAS of the intelligent performance presentation system, the agents at different layers are different, primarily reflected in that “intelligence increases with the level, while precision decreases with the level.” Agents at the same level are also different, mainly due to the complex operating environments faced by each presentation subsystem, where control agents in various subsystems need information from the entire performance operating environment when making judgments. This performance operating environment information is achieved through a standardized open performance presentation system platform, which integrates the environmental information of each subsystem, as shown in Figure 4. The figure shows that the performance presentation environment integrates the operating environment information, equipment operation maintenance information, and performance information from various presentation subsystems. If we abstract the functional structure of agents, the differences between agents lie in their decision-making strategies, actionable tasks, and knowledge representation methods. At the same time, agents also share some common characteristics, such as communication methods, executors, and representation of mental states. Therefore, when constructing agents, a common kernel structure (Agent kernel structure) can be defined for all agents by separating different components. An interface is defined on the agent kernel to facilitate the connection of different decision-making methods and functional modules of agents at various levels to the agent kernel, as illustrated in Figure 5.
Each agent consists of a general agent kernel and many functional modules. The agent kernel includes internal data storage, mailboxes, blackboards, processors, etc. Among them, the internal data storage contains information about the agent itself, goal sets, integrated environmental models, etc.; the mailbox provides communication between the agent and the environment as well as other agents; the blackboard enables communication among various functional modules within the agent; the processor completes message dispatching and functional module execution control, etc. Each functional module is a relatively independent entity, which executes completely in parallel once initiated by the processor, coordinating work through the blackboard.
Using this method, agents with a composite structure in the MAS of the intelligent performance presentation system can be easily constructed, incorporating functions such as perception, reflection, communication, modeling, planning, and decision generation into the agent. Communication is the basis of cooperation. When using multi-agent systems for complex performance presentation supervision, the agents integrated into the performance presentation system platform must be able to communicate and collaborate through some agent communication language. Agent communication primarily involves two issues: communication methods and communication languages. In the MAS of the intelligent performance presentation system, communication between agents adopts a message-based communication method, securely encapsulated by RFC6455/HTTPS.
In the MAS of the intelligent performance presentation system, cooperation is a key difficulty and focus of system design. Agents can only achieve the following objectives through mutual cooperation:
-
Improve task completion efficiency through parallelism;
-
Expand the capability range for task completion through resource sharing (information, expert knowledge, equipment, etc.);
-
Use different methods to complete tasks, enhancing the reliability of task completion;
-
Reduce conflicts between tasks by avoiding harmful interactions between them. Therefore, establishing a MAS with cooperative technology is an important means to enhance the overall performance of the intelligent performance presentation system.
An important aspect of multi-agent cooperation technology is the multi-agent cooperation model. From the development process of modeling multi-agent cooperation mechanisms, formal modeling methods based on logical reasoning and modeling methods based on decision theory and dynamic programming are gradually merging, both emphasizing the rational role of agents, with decision theory serving as the medium for this integration. Various types of multi-agent models and application systems have emerged from different angles to adapt to different application environments, including models such as BDI models for rational agents, negotiation models, cooperative planning models, and self-coordination models. Among them, self-coordination models can adapt to dynamic real-time supervision and optimization of complex performance presentation systems, and their methods can be referenced to establish the agent cooperation model in the intelligent performance presentation system.
Excerpted from “Performance Technology” 2016, Issue 11, Hou Chunhai, Zhou Baoning, Zhou Qilin, “Intelligent Performance Presentation System Based on Multi-Agent.” Please cite: Performance Technology Media. For more detailed content, please refer to “Performance Technology.”
