Let AI Evaluate AI: Building an Automated Operation Agent System for Intelligent Customer Service

The rapid development of large models and computing power has driven transformation in many industries. Even though AI is still in its early stage of “autonomous action,” or the “human assistance” phase, its enormous potential and evolutionary path are relatively clear.

The customer service field, as one of the earliest areas to utilize “intelligent capabilities,” has also gained new vitality with the development of LLMs.

Let’s first take a look at the development of intelligent customer service.

1. Traditional Robot Customer Service Based on NLP

The previous generation of “robot customer service” built on “NLP,” “rule engines,” and “knowledge bases” brought a “leap” to the more traditional “human customer service.” However, it faced criticism due to poor intent understanding (multi-intent understanding is more difficult and often requires targeted training), high maintenance costs (high cold start costs and knowledge maintenance costs), and weak dialogue capabilities (almost zero generalization ability, stiff responses, and poor anaphora resolution), which brought many complaints to “robot customer service.” The key operational tasks during this phase of customer service (This refers to the operation of robot customer service, not scheduling, training, quality inspection, assessment, etc. for human customer service) included:

Knowledge base construction (feeding FAQs): Organizing common customer questions into “question-answer” pairs;
Synonym and rule configuration: Configuring a large number of possible keywords and synonyms for each question. For example, configuring “how to refund” with “want to return goods,” “how to refund money,” “not needed anymore,” etc.;
Dialogue flow design: Designing complex, tree-like dialogue flows (SOPs, decision trees, etc.), for example, “User selects A -> jumps to flow A1 -> asks for information A2…”;
Monitoring and maintenance: Monitoring which keywords are triggered, which questions are not recognized, and then continuously patching and adding new rules;

Language is infinite, but rules are limited. The customer service operation team is always chasing the ever-changing new ways users ask questions, caught in a “cat-and-mouse game,” exhausted. Even more heartbreaking is that no matter how much optimization is done, the intelligence level of the robot is limited by the scale of the rule base from the beginning of its design, making it unable to handle out-of-rule questions.

2. Intelligent Customer Service Based on RAG

The rapid development of large models has redefined “robot customer service,” making a significant leap towards “intelligent customer service.” Our initial application of LLMs in customer service started from RAG (RAG has been over-discussed, but it still needs to be mentioned), aiming to improve the accuracy of document Q&A and reduce the operational costs of knowledge.

Why is RAG effective? Because it is neither purely

nor is it

but rather finds the most relevant content from the “knowledge base,” packaging the “retrieved information” and the “user question” into a super prompt, with LLM acting as an “expert assistant” rather than the “expert itself.” It provides more accurate knowledge as input and can express it more naturally after being refined by LLM.

In retrieval (Retrieval), as the lifeline of RAG quality, the introduction of vectorized retrieval compensates for the shortcomings of traditional keyword and search engine retrieval (ES method), effectively improving recall rates.

ES method:

Vectorized method:

RAG often integrates both:

In enhancement (Augmentation), the retrieved information segments and user questions are carefully assembled into a prompt that LLM can understand, laying the foundation for the generation phase. This includes two aspects:

Information assembly:

On the engineering side, the candidate knowledge fragments obtained during the retrieval phase are sorted by relevance and combined with historical dialogue content to form a complete “contextual background information”;

Prompt engineering, which is the soul of the “augmentation” phase, usually includes the following parts:

System role: Define the identity of LLM. For example, “You are a professional customer service assistant, answering questions strictly based on the provided background information”;
Background information/instructions: Clearly inform LLM of the source and rules of the answer. For example, “Please strictly answer the final question based on the background information marked with “` below. If there is no answer in the background information, please directly say ‘Based on the existing information, I cannot answer this question,’ do not fabricate information”;
Background information content: The aforementioned context;
User question: Insert the original user question;
Output format requirements (optional): Request LLM to output in a specific format (such as Markdown, bulleted lists) or require it to cite sources.

In generation (Generation), the assembled prompt is sent to the large language model. LLM will perform the following core operations based on its powerful language understanding and generation capabilities:

Understanding instructions: Recognizing its role and answering rules;
Relating information: Locating the parts directly related to the question in the provided background information;
Summarizing and generating: Inducing, summarizing, and reorganizing the found information, expressing it in a more natural and fluent language rather than simply copying and pasting.

After introducing RAG, our robot customer service main workflow is as follows:

The capabilities of RAG have allowed the knowledge source of robot customer service to leap from the previously relatively singular FAQ dimension to the entire document dimension (doc, pdf, ppt, tables, etc.), which has a significant impact on the robot knowledge Q&A scenario.

Although LLMs at that time were not as powerful as they are now, and the amount of context tokens they could support was limited, many tasks (intent understanding, multi-turn support, fine-tuning of RAG results, etc.) still required collaboration between the engineering side and small models to complete. However, this was enough to bring significant changes to the work content of customer service operations and reduce the costs of cold starts and knowledge maintenance to some extent. After all, uploading knowledge documents is much easier than configuring a series of FAQs. Customer service operations can focus their energy on building business SOP processes and keeping knowledge documents fresh.

3. AI-Native Intelligent Customer Service

LLMs have experienced a brilliant debut at the end of 2022, the hundred-model battle in 2023, and the emergence of AI applications in 2024, ushering in a prosperous development of intelligent agents in 2025. As the capabilities of foundational models improve, more complex logic can be entrusted to LLMs for understanding, and even external tools can be directly called through MCP, utilizing the Function-Call capability to dynamically leverage local service data, allowing “traditional R&D” engineers to directly build a complete robot customer service workflow by calling foundational models (for example, based onBailian), enabling the implementation of AI-native intelligent customer service.

The main workflow of AI-native intelligent customer service:

In the workflow, the places where models are used have switched to LLMs, with different sizes of LLMs used depending on the scenario. For example, smaller models (7b) are used for rewriting, while larger models (max, 32b) are used for planning and decision-making. Additionally, because it is a dialogue scenario, it is sensitive to time consumption, requiring a balance between LLM selection and response time. Generally, the larger the model parameters, the better the effect, but the longer the response time will be.

Due to the power of foundational models, we can let LLM infer and judge more content (evolving significantly compared to before), for example, complex business logic can be described in natural language as “If xxx, then xxx” format, combined with contextual information, allowing LLM to decide which logical branch to take. We call this “intelligent SOP. Intelligent SOP (still has a lot of optimization space) needs to dynamically call content outside of LLM, which can be achieved through Function-Call.

For knowledge-based Q&A, RAG is still indispensable and is continuously evolving:

4. How to Evaluate Dialogue Effectiveness

Building a dialogue workflow based on AI-native is fine for a demo, but if it is to be commercially delivered, we must focus on the actual effectiveness. The key content of this article is how to observe its effectiveness.

Although many stages of the workflow use LLMs, we focus on the end-to-end effectiveness of robot responses. Simply evaluating LLM itself is not our goal.

The traditional approach is for operational personnel to specify an evaluation set or extract service records from the production environment, manually annotating to see the effectiveness. If problems are found, they need to be recorded separately and find the technical team to locate which part of the Q&A workflow may have caused the BadCase. The repair plan may require operational intervention or may be sufficient with technical intervention, and after repairs, re-evaluation is needed to see the results. The entire process is time-consuming and labor-intensive.

If robot dialogues can be implemented based on AI-native, can the evaluation of dialogue workflow effectiveness also utilize AI-native capabilities to save on manual participation costs?

The answer is yes. Through continuous exploration and practice, we have built a “evaluation-diagnosis-optimization” three-in-one operation Agent platform (avertical Agent) that, combined with knowledge Agent and dialogue control Agent, has constructed an intelligent AI-native service closed-loop system, achieving continuous evolution of customer service robot service quality.

In the actual application with KA customers, the accuracy rate of discovering service records of BadCase is over 85%+, and the accuracy rate of root cause classification and optimization suggestion generation is over 80%+. The operational colleagues of robot customer service can operate based on the output results of the operation Agent, saving a significant amount of costs. At the same time, this capability has also been applied to ToB products, with the overall accuracy of evaluation capabilities currently reaching over 85%+.

5. How to Implement Evaluation

First, confirm the evaluation capability goals, identifying the accuracy of AI-native customer robot responses, classifying the root causes of BadCase, and being able to automatically generate optimization suggestions for BadCase, with the capability for automatic execution of optimization suggestions.
To achieve the above goals, we need to decompose the entire dialogue workflow and drop key information according to the granularity of functional modules (DB or logs), allowing us to link the key workflow from receiving the Query to producing the Answer based on the service record ID.

Secondly, define the main framework for effect evaluation:

Among them:

Building the evaluation set to adapt to different evaluation situations. We provide the ability to automatically generate evaluation sets based on knowledge content, as well as the ability to upload original evaluation sets in file format; at the same time, for scenarios that have already used AI-native robot customer service, we support dynamically filtering service records from the production environment as evaluation sets based on various combination conditions.
Our approach is to first evaluate the robot’s response content based on the evaluation rules in an end-to-end manner (i.e., hitting the rules), and those that do not meet expectations are considered BadCase. Identifying BadCase narrows the target range, providing possibilities for further root cause classification and optimization suggestion generation. The definition of rules is crucial, directly affecting the output of BadCase. We evaluate from several dimensions such as semantic relevance, expression quality, content compliance, and information completeness, with each major dimension containing its own more detailed sub-dimensions.
Root cause classification will directly point to the core of BadCase. We define more than a dozen categories of root causes, including rough recall detection issues, fine-tuning issues, generation error issues, generation hallucination issues, etc. The more detailed the definition of root cause categories, the more accurately we can identify and solve problems.
Once the root cause of BadCase is identified, some categories can automatically generate optimization suggestions and execute them automatically. For example, for root causes related to lack of knowledge, we can automatically trigger retrieval of related knowledge from the internet based on the dialogue context; for rough recall omissions and fine-tuning omissions, we can add similar questions to the identified fragment information to increase the probability of being recalled when encountering similar Queries. In practice, we have found that automatically generating similar questions is easier to implement.
Based on AI-native dialogue workflows, to ensure performance and experience, the LLM parameters used are not particularly large. Our evaluation capabilities are also based on AI-native and can serve as a “referee” to evaluate the same AI-native dialogue workflow (accuracy over 85%+). Two important characteristics of LLMs are that we use larger-sized LLMs in many stages, and we have specifically enabled thinking modes, which allows LLM’s reasoning and judgment capabilities to have greater room for development. At the same time, the related content provided to LLM (dialogue context, reference knowledge, business rules, etc.) is also more comprehensive, providing a more solid foundation for LLM’s reasoning.

Finally, proposing the idea and theory of building a platform is not difficult, especially in AI-native scenarios. However, to achieve the goals, we must continuously discover, think about, and solve problems in practice to improve the final evaluation effect.

6. What Challenges Have Been Encountered

The process of building an operation Agent for dialogue effect evaluation is painful. How to ensure rules work accurately, how to design context, and how to reduce LLM’s erratic behavior are some experiences we have accumulated through trial and error, which I hope can serve as a reference.

6.1 Positive vs. Negative Evaluation

To evaluate whether the robot’s answer meets expectations, we first need to define what the “rules” are. Our initial idea was to create several general rules, each with a score, allowing LLM to judge whether the Answer hits the rules based on the input Query and the robot’s output Answer. Each rule item was assigned different scores based on weight, for example, the “semantic relevance” dimension had the highest score (50 points), while the “expression quality” dimension was set lower (20 points), with the total score of all evaluation dimensions adding up to 100 points.

The scoring judgment here is not to let LLM only do “positive judgment” or only do “negative judgment,” but rather a mixed approach. Purely “negative” judgment would cause LLM to overly focus on defects, ignoring overall value; while purely “positive” evaluation is too lenient and may overlook issues.

We adopted a “mixed strategy” approach, conducting layered evaluations that include clear red line deduction points while also allowing LLM some leeway to make comprehensive judgments within a certain score fluctuation threshold. LLM’s thought process becomes “find strengths → find weaknesses → comprehensive weighing,” which is closer to a human’s overall assessment.

6.2 Strong Dependence on Business Scenarios

With the evaluation rules established, we found in practice that for some service records, the evaluation always fails to meet expected results. For example, when a user inquires about the “course cancellation policy,” the robot needs to further clarify which type of course it refers to in order to provide corresponding content, as different types of courses have different rules (e.g., type “A” courses can be canceled 2 hours before class, while type “B” courses require 3 hours in advance). If the operation Agent does not know the details of different types of rules or the customer’s true demands, it will lead to misjudgment.

Therefore, before the operation Agent judges whether the dialogue content of service records is a BadCase, we need to do two things: 1. Identify the customer’s demands; 2. Obtain strongly related business content based on customer demands as reference knowledge for LLM.

6.2.1 Customer Demands

How to accurately identify customer demands? Relying on LLM to extract from the current Query content may have accuracy issues. If it involves multi-turn dialogue, it is even more necessary to combine historical dialogue content to extract demands, as demands may change, or multiple pieces of content may need to be combined to identify the complete demand (as in the previous example).

More importantly, some business scenarios are quite complex. For example, a fitness company includes multiple fitness brands, each with similarly named courses (e.g., personal training classes, group classes), and each course has similar sub-scenarios (e.g., course reservation, course cancellation, etc.), and the descriptions of these sub-scenarios can be quite close, affecting the identification of customer demands.

Customer inquiries are varied, while a business’s customer service scope is clear and enumerable. We designed two levels of associations: business categories and business scenarios.

Business Categories are similar to the concept of “business lines,” such as Ant Huabei business, Jiebei business.

Business Scenarios are secondary classifications of business, such as “how to activate Jiebei service,” “how to repay Jiebei,” etc.

Through analysis of service record data and communication with business colleagues, we sorted out the above KA fitness company’s 7 business categories and over 40 business scenario information, which were used as input content in the Prompt, allowing LLM to map the Query information with category/scenario information while considering the continuity of historical demands and the emergence of new demands, thus identifying the customer’s current latest demand.

For example, in the following case,

{  "customerQuery": "transfer",  "robotAnswer": "Could you please clarify which type of course you are inquiring about transferring? For example, personal training or another type?"}, {  "customerQuery": "membership card",  "robotAnswer": "Are you inquiring about a specific issue regarding the transfer of the membership card?"}

If the user simply asks “membership card,” it is very easy to misidentify the intent as a membership card inquiry. After processing through the above logic, we can effectively identify the user’s intent, providing a guarantee for the accuracy of subsequent root cause classification and optimization suggestion output.

{  "customerDemand": "membership card transfer inquiry",  "brand": "**fitness membership card",  "scene": "membership card transfer scenario",  "thought": "The customer's current question is 'membership card,' combined with the historical dialogue where the customer repeatedly mentioned 'transfer' and 'membership card,' and the customer service clearly asked 'Are you inquiring about the membership card transfer issue?' It can be determined that the customer's demand focuses on the membership card transfer. Based on the category information list, 'membership card' belongs to the business category of xx fitness and has not mentioned other brands, brand xxx, combined with the business category recorded as 'xxxx.' The scene matches the core words of the scene information list item 28 'membership card transfer scenario,' 'transfer card,' 'membership card transfer.'"}

Business category and business scenario information (or referred to as “intent layering system”) can be extracted through automation, but manual intervention is also needed for freshness:

6.2.2 Reference Knowledge

Evaluation requires viewing every link in the dialogue workflow with a “skeptical” perspective, such as whether the knowledge retrieved by RAG is okay, and whether there is a possibility of missing some fragment information?

This relates to the sorting of the dialogue workflow mentioned above. We have agreed with the knowledge Agent and dialogue Agent to persistently save the output results of each intermediate link. For example, when the knowledge Agent is retrieving, it needs to save the rough recall results and the fine-tuning results. The operation Agent will evaluate the result data at each stage. If it finds that a candidate fragment did not appear in the rough recall results, it will classify the root cause as “rough recall omission”; or if the target fragment exists in the rough recall data but disappears in the fine-tuning results, it will classify the root cause as “fine-tuning omission.” Based on this logic, there is now a primary question: how to know all the target knowledge fragments associated with the Query?

Queries or service records are strongly bound to a specific customer service robot and belong only to one robot. Our initial idea was to use all knowledge documents bound to the current robot as candidate knowledge, using Query, historical dialogue, and extracted business category/business scenario/customer demand as comprehensive input information to allow LLM to verify which fragment content among the candidate knowledge is the target fragment knowledge strongly associated with the Query.

This method is applicable when the knowledge materials bound to the robot are relatively few, checking each fragment of the knowledge bound to the robot to find the target fragment. However, when the knowledge materials bound to the robot are numerous, it becomes unsuitable due to slowness!

In a normal RAG Q&A workflow, the top N knowledge content is recalled from the knowledge base based on set thresholds. We have slightly lowered the threshold to expand the number of top N, aiming to ensure that as much content related to the Query as possible is retrieved. Moreover, this recalled quantity is no longer all document fragment information, which also improves performance.

6.2.3 Exemptions

Due to the specificity of some business scenarios, customer Queries may trigger special logic after hitting the rules, such as transferring the robot to a human or the robot outputting fixed scripts. Even if the content of these robot responses does not resolve the customer’s demands, in such special scenarios, their responses meet business expectations and should not be marked as BadCase, but rather exempted.

Initially, this exemption judgment was integrated with the logic of discovering BadCase in a single Prompt fed to LLM, where dialogue content that met exemption rules was not marked as BadCase. However, in practice, we found that recognition could be biased. To avoid “false positives,” we extracted the exemption judgment into an independent logic, first conducting exemption detection, then identifying BadCase.

6.3 LLM’s Adversity

The large model itself is also a probabilistic model, meaning that even if a “certain” prompt is given, the results it outputs have “randomness.” With the previous rules and data, we encountered issues during BadCase identification due to this “randomness,” which manifested in two aspects:

(1) Using the same LLM (leading LLMs in the industry), with the same prompt, multiple calls yield different results;

(2) The same prompt with different LLMs (leading LLMs in the industry), also produces different results, sometimes even opposite results;

6.3.1 The First Problem

For the first problem, the core lies in the “random sampling” of LLM. Randomness itself is not an issue; it is actually the source of LLM’s reasoning and diversity. The generation of results by large models (taking text content as an example) is an autoregressive process, generating one token at a time. When you input a prompt, the model executes the following steps:

Step 1: Process the input content;
Step 2: Calculate the probability distribution of all possible next tokens;
Step 3: Select one token from these distributions as output;
Step 4: Append this newly generated token to the original context and repeat the above process to generate the next token, and so on.

The key lies in steps 2 and 3.

In step 2, LLM calculates scores (logits) for each possible word (let’s consider tokens this way) and then converts these scores into probability distributions using a function called Softmax. For example, if the prompt is “Today’s weather is nice, let’s go to…”, when LLM predicts the next word, the original scores (logits) calculated and the probabilities after applying Softmax might look like this:

If there were no randomness, the model would always select “park,” resulting in identical outputs every time, which would be very rigid. To increase randomness, LLM uses two important parameters:Temperature and Top-P

Temperature (temperature)

Temperature is a parameter applied before the Softmax calculation, directly reshaping the probability distribution (my personal shallow understanding):

Probability = Softmax(original score / Temperature)
Low temperature (e.g., 0.1): Dividing by a very small number amplifies the gap between high and low scores;

Effect: Increases the probability of high-probability words (like “park”) and decreases the probability of low-probability words (like “fly”). The model output becomes more certain and conservative;
In the example, the probability of “park” might rise from 65% to 95%, while other words are severely suppressed;

High temperature (e.g., 1.5): Dividing by a number greater than 1 reduces the gap between scores;

Effect: Flattens the probability distribution. Low-probability words (like “eat,” “fly”) have increased chances;
In the example, the probability of “park” might drop from 65% to 40%, while “walk” rises to 30%, “eat” to 20%, and “speeding” to 10%. The output becomes more diverse, creative, but also more unstable.

Top-P

Top-P is a filtering mechanism applied after the Softmax calculation. It sorts candidate words by probability from high to low and retains only the smallest set of words whose probabilities sum just over the Top-P in P (e.g., p=0.9). The Top-P process is as follows:

Sorting:park(65%), walk(24%), eat(9%), speeding(2%)……;
Cumulative probability calculation:park(65%), park+walk=89%, park+walk+eat=98%(already > 90%);
Filtering: Only retain {park, walk, eat}. The low-probability “speeding” is eliminated;
Re-normalization: Rescale the probabilities of these three words to sum to 100% (e.g., park ~66%, walk ~25%, eat ~9%);
Finally, random sampling is performed from this new, filtered set.

Thus, as long as Temperature > 0 orTop-P < 1.0, the model faces random results at each step, and a complete result generation is a sequential process where each step’s random choice affects subsequent steps:

P(full text)=P(x1)⋅P(x2|x1)⋅P(x3|x1,x2)⋯P(xn|x1,……,xn−1)

Every small random difference at each step is exponentially amplified through this chain rule, leading to significant differences in the final result.

6.3.2 The Second Problem

Why do different LLMs produce different outputs with the same prompt (even if they are all leading LLMs in the country)? This is mainly due to differences in tokenization and training parameters.

Tokenization Differences

LLMs do not directly understand the text content we input; they first split the text into “tokens.” Different models have different tokenizers. For example, the tokenizer A might split the word “unstable” into "un" and "stable", while tokenizer B might treat it as a whole "unstable". This leads to different initial input sequences received by LLM, and naturally, the subsequent reasoning calculations differ.

Training Parameter Differences

The core of LLMs consists of hundreds of billions or even trillions of parameters, which are learned from vast amounts of data during training. Different LLMs have different training processes and reference data. These parameters determine how LLM processes the input tokens obtained through “tokenization” and calculates the “original scores (logits).

Additionally, LLM’s instruction fine-tuning and RLHF vary widely, which is also a reason for the differences in output results among different LLMs.

6.3.3 How to Solve

For the first problem, when calling LLM, we can try adjusting the Temperature and Top-P values (the Baidian SDK supports dynamic settings), which, while not completely avoiding the issues caused by randomness, can reduce the probability of divergence. For the second problem, we adopted a method of adversarial multiple LLMs, meaning that during BadCase identification, we execute the same prompt through two or three LLMs, using statistical methods to obtain the final result (as long as the execution time of LLM is not very sensitive).

Example of LLM judging a BadCase:

{  "qwen3-235b-a22b" : "{\"score\":30,\"thought\":\"1. **Content Compliance**: The customer service response did not directly address the customer's request to suspend the card due to store renovation, and did not correctly reference the rule in the reference knowledge that special situations (such as renovation) require contacting a human or the store manager, violating the compliance requirement to transfer to a human in special scenarios, deducting 20 points.\\n2. **Semantic Relevance**: The customer's clear demand is to suspend the card for a week, but the customer service did not provide an effective solution (such as guiding the suspension path or confirming eligibility), only repeatedly guiding to select orders or transfer to a human, failing to solve the actual problem, deducting 30 points.\\n3. **Expression Quality**: The language structure is reasonable, but it does not clearly state the suspension rules, leading to vague information, deducting 5 points.\\n4. **Information Completeness**: It did not mention the conditions for suspension (such as needing to note the reason, review time, etc.) or the operation path (such as the link to apply), missing core information, deducting 15 points.\\nOverall score: 100-20-30-5-15=30"}