This article from the technical community, “AI Agents: An Exploration and Introduction from Concept to Practice,” explains some discussions about AI Agents in large models, from which we can understand some basic knowledge about AI Agents.
Related historical articles on large models include:
“Top-Level Architecture Design Experience of Enterprise Large Model Applications“
“Bare Metal GPU vs Virtual GPU: How to Choose?“
“Understanding NVIDIA CUDA“
“How to Choose Between GPU and CPU?“
“Discussion on Regulatory Data Security of Large Models”
“Understanding GPU Technology in One Article”
“Opportunities and Challenges of Large Model Applications in Finance”
“How Much Resources Are Needed to Build a Large Model from Scratch?”
-
Viewpoint 1
The financial industry is a field densely populated with artificial intelligence application scenarios and is one of the best areas for the implementation of large model technology. Utilizing this opportunity can accelerate the development of financial digitalization and intelligence, reshape existing business processes, and change the industry landscape.
1. Financial Risk Management. In the financial industry, risk control is the core of the core. Traditional risk control methods often rely on human experience and limited data analysis, making it difficult to comprehensively and accurately assess potential risks. Large model technology can be used to build more accurate and comprehensive risk models, helping banks assess and manage market risk, credit risk, operational risk, etc., providing more precise risk predictions and decision support, which helps banks formulate effective risk management strategies.
2. Quantitative Trading. Traditional quantitative trading has always been a high-end play in the financial sector, but high costs and complex technical thresholds deter many investors. Large model technology can be applied to the development and execution of quantitative trading strategies. For example, a bank’s large model can analyze massive market data and economic indicators, identify potential trading opportunities and trends. Through complex algorithm models and optimization strategies, the bank’s large model can automatically execute trading orders, achieving high-frequency trading and arbitrage operations. More importantly, the bank’s large model can also adjust trading strategies in real-time based on market changes, ensuring that investors can always seize the best investment opportunities. In this way, even ordinary investors can enjoy the intelligence and efficiency brought by quantitative trading, which helps improve trading efficiency, reduce trading costs, and enhance trading stability.
3. Financial Fraud Detection and Prevention. Large model technology can be applied to financial fraud detection and prevention. By analyzing users’ transaction data, behavioral patterns, and historical records, it can identify potential fraudulent behaviors and abnormal transactions, improving banks’ ability to identify and respond to fraud risks, and protecting the safety of customers and the financial system.
4. Intelligent Customer Service. Large model technology can be used to build intelligent customer service systems, enhancing customer satisfaction and loyalty through smooth human-machine dialogue services.
5. Precision Marketing. Large models can serve as “digital employees,” providing automated customer service experiences close to real human interaction, reducing labor costs and enhancing service efficiency. Large models analyze customer data to achieve precision marketing, improving marketing response rates and conversion rates. By constructing personalized profiles, personalized investment advice and portfolio configurations can be generated based on individual investors’ preferences and risk tolerance, assisting investors in making more informed decisions.
6. Data Governance. Through machine learning and deep learning technologies, large models can provide more accurate data analysis and predictions. They can automate repetitive tasks, reduce human intervention, and improve work efficiency.
However, during use, considerations must be made regarding the challenges faced by large models, such as high computational demands, high training and inference costs, poor data quality, privacy and security issues, and the need for better alignment between large models and the financial industry’s requirements.
-
Viewpoint 2
In fact, the market application potential of large models in the financial industry is quite rich, with specific application scenarios as follows:
1. In customer service agents: we can use large models to understand customer needs, generate constructive investment advice, and also serve as virtual assistants for real-time interaction and responses to customer inquiries, providing a friendly user experience.
2. In risk management: using large models to process unstructured data (such as news, social media sentiment) and structured data (such as financial statement data, transaction data, etc.).
-
Viewpoint 1
General needs can be decomposed into five levels: value, process, activity, task, and step.
Value corresponds to enterprise strategy, which is rarely involved in the implementation process. We will explain the following four points combining process, activity, task, and step.
Process: Completed by multiple organizations or roles working together, each process has a clear goal.
Activity: Processes usually consist of multiple business activities. Activities are generally completed by a single role or by people of the same role.
Task: Tasks make up activities and are related to scene conditions.
Step: The components of tasks, involving a single role or personnel, generally unrelated to scene conditions.
For example, in a banking scenario, the marketing product of a customer manager is a business activity. This business activity can be decomposed into: pre-sale customer demand exploration, in-sale product personalization configuration, and post-sale marketing effect tracking, etc. These business tasks can be further decomposed into multiple business steps, such as customer profiling analysis, answering common product questions, identifying customer needs, product personalization configuration, intelligent form filling, customer product holding period, product purchase cancellation rate, conversion rate, etc.
Business tasks can be subdivided into simple and complex business tasks based on business complexity. For simple business tasks, we can quickly implement them through workflow orchestration. For example, in customer profiling processing, workflows can quickly and accurately extract specific information from the knowledge base through RAG, such as asset liabilities, product preferences, risk aversion, etc., and automatically combine them with customer ratings and historical customer behavior patterns to present a dynamic customer profile.
For complex business tasks, we can achieve functions such as question rewriting, expansion, and counter-questioning through dialogue flow + memory components, allowing for a more precise understanding and clarification of task requirements, enabling AI Agents to better understand user intentions and provide personalized solutions. For instance, in product profitability consulting, through dialogue with customers, assess their willingness to purchase and loss tolerance, investment experience, etc., ensuring rapid and accurate customer classification to improve marketing efficiency.
Integrating AI Agents with the IT systems currently used by enterprises is also a way to meet business needs, commonly including CRM, ERP, and some analysis systems, such as automatic data entry for sales, order processing, etc., thus improving sales efficiency. When a customer places an order, the Agent can automatically retrieve customer information from the CRM system, check inventory from the ERP system, and generate orders in sales automation tools.
For example, during marketing effectiveness analysis, the Agent analyzes marketing data and finds significant changes in a certain performance indicator, which can promptly alert management for timely strategy adjustments and actions.
What has been discussed above is just some enhanced uses of Agents. In reality, many low-code platforms that have implemented business processes can completely embed Agents for use, achieving a process engine-style editing.
-
Viewpoint 2
Currently, no universal framework similar to Springboot in Java development has emerged in the application field of large model Agents. The reason for this is likely due to computational power being the key: on one hand, the computational power required for model iteration and continuous development of new models is enormous, making it so that even after years of evolution, open-source model capabilities are still insufficient to reach AGI; on the other hand, the computational power required for large model inference far exceeds that of traditional Java applications, and for complex business needs, its costs have not yet shown overwhelming advantages compared to manual labor.
Compared to traditional business needs decomposition, large model Agent applications also need to pay extra attention to the capability boundaries of the Agents themselves. For example, for business needs involving mathematics and reasoning, it is necessary to preemptively assess the risks of unreliable results from large models in collaboration with business parties; also, due to the large amount of input and output from large models, the response latency of Agent application systems is often significantly larger than that of traditional Java applications. It is necessary to clarify the time-sensitive aspects of the business with business parties in advance and design the system accordingly.
-
Viewpoint 3
Generally speaking, the process from demand to product is a systematic project. Decomposing enterprise needs into Agents and processes essentially simplifies complex problems into multiple manageable subdomains. From the perspective of implementing processes, the following steps can be referenced:
1. Understand the needs and identify the processes: First, clarify what the implementation goal is. Then, for specific goals, decompose the entire business process into specific steps, and clarify the input, output, and processing logic of each step.
2. Design Agents and interaction flows: Based on each step of the business process, determine what Agents are needed to complete the tasks. At the same time, each Agent is responsible for a relatively independent function, assigning each task to the most suitable Agent and defining its input and output.
-
Viewpoint 1
1. RAG Enhancement: RAG is a technology that combines information retrieval and text generation, guiding text generation by retrieving relevant information from external knowledge bases. This method improves the model’s accuracy and reliability in handling complex problems, suitable for Q&A systems, document generation, and other tasks.
2. Memory Supplement: Incorporating memory information into user prompts, constructing information that has judgment significance for the business, such as supplementing subjects, objects, and scenes based on contextual semantics, to help RAG achieve greater effects.
3. Code Optimization: For user rewrites, completing missing entities in statements, or rewriting erroneous semantics to improve RAG success rates.
-
Viewpoint 2
To maintain the stability of AI Agents, the system framework needs to fully utilize and organically combine various mainstream technologies, mainly including:
RAG Technology: RAG is an important means to enhance the credibility of large model output content, significantly alleviating the hallucinations in endogenous knowledge of large models by providing external credible knowledge, effectively improving system stability.
Planning, Decision-Making, and Reflective Architecture: The ReAct model has become the mainstream design concept for current AI Agent architectures. By requiring large models to self-reflect on potential errors, it can effectively correct hallucinations in outputs, enhancing system stability. However, it is important to note that the architectural design should remain simple, introducing reflection mechanisms only at critical points to avoid overly lengthy reasoning chains that could affect model performance or even lead to infinite loops.
Formatted Input and Output: Although large models are seen as interacting in natural language, this mainly refers to the final results output to users. In AI Agent applications, there are substantial interactions between models and between models and computer systems, and formatting or even coding input and output is a very effective solution for hallucination elimination. Models can be fine-tuned to endow them with formatted input and output capabilities, enhancing system stability.
-
Viewpoint 3
In practical scenarios, maintaining the stability of AI Agents and ensuring that the output content of the underlying large model consistently maps to the designated intent of the Agent is a complex and multi-faceted challenge process. This not only involves technical implementation but also requires a deep understanding and management of user interactions. We can start from the following points for reference:
1. Optimization of Data Augmentation: For example, using high-quality, accurately labeled training data to ensure that the underlying model can learn stable relationships between intent and output. At the same time, ensure that the dataset covers various expressions and contexts to improve the model’s robustness and ability to handle different user inputs. Additionally, using data augmentation techniques (such as synonym replacement, sentence restructuring, etc.) to generate diverse training samples further enhances the model’s adaptability.
2. Iterative Fine-Tuning of the Model: For specific intent target requirements, regularly fine-tune the model using the latest interaction data to keep the model updated and adaptive, thereby making the model’s intent mapping for these specific scenarios more comprehensive and accurate.
-
Viewpoint 1
1. Clarifying Needs is the First Step: Enterprises should first conduct in-depth discussions on their business scenarios, identifying which specific problems need urgent solutions and what future development plans exist. This helps enterprises form a demand list, clarifying goals and avoiding unnecessary functional redundancy, making the selection process more efficient.
Short-term Goal: If enterprises wish to quickly enhance the intelligence level of existing systems in the short term, they can choose the AI Embedded model.
Medium to Long-term Goals: For businesses requiring long-term continuous improvement and relying on user-AI interaction, they can choose the AI Copilot model.
Full Automation: If enterprises wish to achieve high automation of business processes and reduce human intervention, they should consider the AI Agent model.
2. Technology and Resource Assessment: Enterprises should assess their technological reserves and R&D capabilities to choose the appropriate AI model.
This includes evaluating the enterprise’s own technology reserves, including existing algorithms, data resources, computing capabilities, etc. If the enterprise has strong accumulations in these areas, they can consider adopting more advanced AI models, such as large-scale pre-trained models or AI Agent technology. If the reserves are not very rich, they can consider the MASS model provided by suppliers.
Research and deployment of AI large models require significant financial investment, especially for models that require large amounts of data for training, so enterprises should choose the appropriate AI model based on their financial capabilities.
3. Usage Scenarios and Applicability: The application of AI large models varies across different industries. Therefore, understanding the actual usage scenarios and applicability of the models is particularly important. Consider the following aspects:
Industry Characteristics: Some models have unique algorithm optimizations and data processing capabilities in specific fields. When selecting, ensure that the chosen model possesses the required specialized functions.
User Feedback and Reputation: Analyze past user feedback and evaluations. User acceptance and reliance on AI are also important factors in selecting AI models.
4. Model Performance and Evaluation: Before officially building the model, consider how to conduct effective performance testing to see if the performance meets the expected requirements.
-
Viewpoint 2
Small and medium banks, when choosing suitable large models and application scenarios, need to combine their business characteristics and resource conditions to find suitable construction paths. The following ideas are for reference:
1. Clearly Define Specific Implementation Goals and Application Scenarios: The first consideration is to select the corresponding scenario based on the business orientation, such as whether to act as an AI advisor or a customer service agent. After all, different business goals may require different technical frameworks.
2. Choose the Right Model: Whether to develop or package based on open-source models or purchase commercial licenses, as well as the subsequent development support and iterative optimization capabilities of these models, all need to be considered.
3. Establish the Technical Stack of the Model: If using open-source models, what language or integration framework to use for packaging later also needs to be weighed, as it relates to future model maintenance issues.
If starting from scratch, the technical reserves required in the early stages cover a wide range of points to consider, including:
1. Application Framework Level: Familiarity with large models and surrounding ecosystems, understanding the associated technology stack of the chosen large model, such as runtime framework platforms, plugins, etc.
2. Infrastructure: If training optimizations are required for the developed model, understanding the underlying technical infrastructure, such as computational demand planning, model lifecycle management, etc., is essential.
3. Data Privacy and Compliance: Ensure compliance with regulatory requirements when introducing large models, establishing mechanisms for data desensitization, encryption, and privacy protection, especially to prevent data leakage risks during model training and usage.
-
Viewpoint 1
The issue of over-interaction of AI Agents has indeed occurred in early applications of Agent frameworks like AutoGPT, but with the continuous in-depth research in the field of AI Agents since 2024, the problem of over-interaction has largely been mitigated.
Some cases indicate that over-interaction primarily arises from the application system’s lack of guidance in task decomposition for large models. When large models operate freely, their thinking may diverge, leading to a deviation from the original task goal. In response, the new AI Agent application development paradigm, similar to the microservices concept in traditional software development, has led to widespread acceptance of multi-Agent collaborative work. Although based on the same large model, different prompts can endow different Agents with different thinking directions, by designing prompts to limit each Agent’s expected input and output, focusing on completing their tasks, forming a service mesh that creates a clear link on the mesh, effectively avoiding over-interaction.
-
Viewpoint 2
Optimizing over-interaction of Agents and improving efficiency is a systemic issue that can be considered from multiple aspects, specifically:
1. Technical Level: Enhance the capabilities of AI Agents themselves, and optimize or adjust related ecological aspects, such as reinforcement learning, knowledge graphs, and other technical means.
-
Viewpoint 1
To facilitate non-professionals, the platform should be fully modular from design to deployment. For example, the AI Agent application platform architecture proposed by our company’s doctoral team can be divided into three layers: large model layer, Agent layer, and application layer.
In the large model layer, users can choose to use the large model APIs provided by service providers or opt for private local deployment of large models. Choosing the API mode can save deployment costs, while private deployment can support higher controllability and customized applications.
In the Agent layer, different Agents can be deployed through containerized modularization, allowing for separate capacity and pricing settings, enabling users to save costs to the greatest extent.
In the application layer, through a core self-developed front-end interface based on the advanced Vue3 framework, users can combine different Agents to design AI Agent applications suitable for their business workflows, meeting various user needs.
-
Viewpoint 2
Usually, from the perspective of “usability,” several aspects can be referenced:
1. Based on Low-Code/No-Code Development Models: Utilizing a visual configuration interface, complex business logic can be achieved merely by configuring parameters.
2. Based on Pre-Made General Templates: Leveraging common business scenario templates, such as intelligent customer service, allows for direct selection and customization.
-
Viewpoint 1
Regarding the security issues of Agents, our company’s doctoral team has been continuously researching and has produced a series of secure and efficient privacy large model solutions over the past two years. Recent research on this issue has also made significant progress.
In consumer applications, the AppAgent produced by Tencent emphasizes considerations of privacy and security in agent architecture design. This framework primarily simulates human operations to automate calls to smart terminals, thus using screenshots as input. When understanding the content displayed on users’ smart terminal screens, special steps for recognizing sensitive content (such as password input boxes, gesture passwords, etc.) are added, returning control to the user for these steps, ensuring the security of consumer agent applications.
In B2B applications, customized reinforcement of large model security can be achieved using NVIDIA’s Nemo Guardrails, with specific situations analyzed on a case-by-case basis.
-
Viewpoint 2
1. Identity Control: Ensure that only authorized users or systems can access and operate AI Agents.
2. Authorization Control: Limit users’ access and operating permissions to AI Agents to prevent illegal activities.
3. Secure Communication: Ensure that data transmission between AI Agents and users or other systems is secure and reliable through encryption, digital signatures, and other technical means.
4. Security Auditing: Monitor all actions and information flows of AI Agents; provide a comprehensive dashboard view for actions, processes, connections, data exposure, information flows, outputs, and responses. Additionally, support immutable audit trails for all interactions and activities of the Agents.
5. Detecting and Tagging Abnormal AI Agent Actions: Detect and tag abnormal AI Agent actions that violate relevant enterprise policies. Attempt to automatically rectify abnormal transactions whenever possible; for those that cannot be automatically rectified, immediately suspend and escalate for manual review and rectification.
6. Immediate Issue Resolution.
-
Viewpoint 3
In practical business scenarios, due to the complexity of the internal mechanisms of AI Agents, their autonomous decision-making, data dependency, and dynamic environmental interactions, AI Agents may face various issues. Therefore, the security of Agents is a comprehensive issue that requires consideration from multiple aspects:
1. Model Perspective: During training, employ adversarial training techniques to enhance model robustness against adversarial samples, preventing the model from being disturbed by malicious inputs. Additionally, comprehensive model validation before deployment is necessary to ensure its security and stability.
2. Data Perspective: Ensure compliance with access mechanisms for specific data and encrypt data during storage and transmission. For core or critical data, anonymization should be performed to protect user privacy and reduce the risk of data leakage.
3. Platform Perspective: Ensure that the underlying facilities on which the application relies are in a controllable state to avoid unnecessary vulnerabilities and risks.
1. Analysis of AI Agent Application Scenarios
2. Practice of AI Agent Demand Implementation
3. Practice of Optimizing AI Agent Stability
However, AI Agents face numerous challenges in practical applications, requiring comprehensive consideration from multiple dimensions, including technology, application, organization, and ethics. To successfully construct and deploy AI Agent systems, solid AI foundational knowledge, rich industry experience, and keen insights into future development trends are essential.
If you find this article helpful, please do not hesitate to click “Like” and “See” at the end of the article, or directly share it in your circle of friends.
