How AI Agents Are Disrupting Traditional Automation

/ 01 /

Predictions on Trends in Automation Development

1. From consumers to knowledge workers, everyone will have their own AI assistant. This will redefine the traditional boundaries between vertical applications, automation platforms, and IT services, creating transformative market opportunities for entrepreneurs. AI assistants will appear in various forms, from auxiliary tools on existing platforms to applications embedded with AI, and various forms of AI agents.

2. Human-computer collaboration is at the core of deploying generative AI solutions. Currently, most application scenarios are in experimental or early production stages, focusing on advisory and assistive workflows. LLMs still lack reliable planning and reasoning capabilities, and areas like memory and context understanding are still under research. In automation platforms, deterministic execution is crucial, and LLMs are used for specific tasks in the ‘design phase’ rather than the ‘execution phase’.

3. Automation is an underestimated challenge. Practitioners are incorporating AI into their solutions and expertise to enhance platform efficiency and user experience. Leading LLM providers are integrating agent modeling, collaboration, and tool access features, enabling users to quickly build AI agents (like the GPT series). Emerging enterprises that are expected to break through need to leverage unique datasets and streamlined user experiences to provide differentiated customer value and reshape workflows.

4. The deployment of built-in AI automation will take a progressive approach from simple to complex, starting with simple tasks and gradually transitioning to complex workflows. The key is to continuously experiment with agents to understand where AI capabilities truly create value and ensure proper ‘scaffolding’ in terms of data, tools, and runtime, making it part of the automation architecture. As AI model capabilities improve, the application scope of AI features can be gradually expanded.

5. Code generation has become a key element in developing generative AI (GenAI) applications and agent automation platforms. Code possesses both text form and clear performance metrics, making it an ideal function for LLMs. Initial versions of programming assistance tools have been widely deployed, and we are seeing more mature AI-driven development platforms emerging. Code generation LLMs will play a critical role in agent architecture.

/ 02 /

The Evolution of Automation Platforms

Automation is a continuous endeavor by humanity. Every knowledge worker is familiar with this unassuming ‘macro’—a shortcut for a set of repetitive commands that can enhance productivity.

Early automation efforts primarily focused on workflows such as quote-to-cash and payroll, where engineers wrote custom code to stitch together these workflows governed by static rules and definitions. These fragile early methods led to the development of the first generation of automation platforms, such as:

Robotic Process Automation (RPA) platforms provide the greatest value in automating repetitive manual tasks. They combine pre-defined workflow libraries with low-code/no-code platforms, helping users build their own scripts. RPA platforms have gradually incorporated AI/ML models to extend their functionality.

Platforms like Workato create a middleware layer to integrate data, application sources, and APIs to connect different resources. This data layer is a key input for the automation engine, and creating a clean interface is the first step in the automation journey.

Low-code task automation platforms offer a set of pre-defined integrations through a simple UI to automate repetitive tasks for knowledge workers and small to medium-sized enterprises.

Various vertical automation methods focus on specific workflows in areas like supply chain, IT operations, and developer ecosystems, as well as customer-facing use cases (such as chatbots for help desks and customer service teams).

While these platforms significantly reduce repetitive work, the guidance required to realize automation value through pre-defined workflows or consulting deployment remains complex. Implementation is also fragile to changes in the enterprise operating environment.

GenAI has the potential to accelerate this automation process, as existing enterprises integrate compelling features into their platforms, builders experiment with new architectures, and researchers strive to achieve the ultimate goal of autonomous artificial intelligence (AGI).

/ 03 /

AI in Automation: Different Participants, Different Methodologies

Automation in enterprises is often a complex task, with some practitioners even referring to it as the complex orchestration of multiple elements executing workflows. With the emergence of GenAI, existing enterprises and startups/scaled companies are seizing this opportunity from different angles.

RPA and task automation platforms bring significant existing advantages, with rich libraries of automation workflows and experience in handling complex workflows in enterprises. GenAI offers an opportunity to address vulnerabilities and guidance issues through a simplified user experience.

Applications like Microsoft 365 and Notion embed AI directly into platforms and user workflows to help complete tasks, provide suggestions, and generate content to assist users’ workflows.

Native AI approaches rethink automation from the perspective of applications or workflows, using first principles. In applications, new-generation productivity tools like Swimm and Writer stunningly demonstrate the transformative impact of generative AI on work patterns. Similarly, many vertical applications from sales, marketing, legal, and finance use GenAI capabilities to streamline complex workflows.

LLM providers and startups are paving new paths in automation, leveraging agents to harness generative AI capabilities for executing simple tasks. Other approaches combine LLMs with necessary ‘scaffolding’ to address complex workflows and applications. Agent automation is currently a focus of ongoing innovation and research, with developers extensively experimenting with models, architectures, and tools.

RPA and Task Automation Platforms

Current generations of automation platforms have actively adopted newer machine learning and artificial intelligence models as part of their platforms. Below is a brief overview of the current state of these platforms:

User Interface connects to low-code studios, where users can build, deploy, and validate automation. This interface is also used to monitor performance, track usage of each strategy, and even measure the return on investment of the automations they create.

iPaaS plays a critical role as middleware, aggregating data from applications, data stores, and event streams to create efficient interfaces to the automation layer.

Automation Layer allows users to select from pre-built scripts, predictive machine learning models, and tool libraries using templates in the studio, or execute new workflows. Some common use cases include:

· Extracting data from unstructured sources like images or emails and filling out forms.

· Observing humans (e.g., reading screens, tracking keystrokes) to generate repeatable workflows or suggest potential new automations.

· Extracting data from inventory systems and creating predictions using ML models.

Existing enterprises are using GenAI to simplify user engagement and provide new workflows, such as:

· Inputting tasks like “sales exploration,” where the co-pilot translates intent and searches the automation library to provide users with a starting point for the task.

· Creating a form and updating it with appropriate fields based on pre-trained templates. Filling out data extracted from various unstructured sources.

· Generating ‘low’ code to create automation based on NL descriptions and test cases to validate the output and workflow descriptions.

AI tools help these platforms leverage their existing advantages (customers and manuals) by assisting users in accelerating time to value. Improved UI/UX helps reduce the consulting guidance typically needed at the start of complex deployments. As LLM capabilities evolve, we can expect the capabilities of RPA and task automation to grow as well.

“In the future, all interactions between humans and the digital world will be realized through AI agents.” – Yann LeCun

/ 04 /

Agent Automation Framework: Copilots/GPT and Agents

It may be helpful to define the terms commonly used in the market to refer to GenAI use cases.

·Copilots are existing applications and platform interfaces based on GenAI that provide users with a simplified way to discover and enhance existing features.

·Agents combine the capabilities of large language models with code, data sources, and user interfaces to execute workflows. Builders are exploring several approaches:

· Building simple wrappers around LLMs or LLMs trained for specific tasks (code generation).

· Expert hybrid architectures with ‘scaffolding’ that combine task-specific agents, predefined code/workflows, and external tools to reimagine applications or automate complex tasks.

·General agents aim to automate any task simply by describing it. For researchers requiring continuous advancements in AI, this remains a long-term goal—learn more in the ‘References and Further Reading’ section below.

No-code Agents/GPT

AI Agents were initially just experiments, with builders like Yohei Nakajima releasing projects like Baby AGI, which are built on local LLM capabilities and can run simple automations. LLM providers now offer compelling no-code platforms that include libraries of external resource plugins to build custom versions of LLMs. For many simple tasks or one-off automations, this can be a quick way to get started.

In this approach, no-code consoles allow users to provide detailed descriptions of tasks or use minimal prompts to guide large models in building agents. LLM providers now offer integrations with data sources and applications, allowing agents to leverage external data as part of their workflows. The agents can also use proprietary data, enhancing accuracy using techniques like Retrieval-Augmented Generation (RAG). APIs introduce external tools like search.

As illustrated, advanced agents can be built using glue code around the capabilities of large models, combining these different elements into a unified agent. LLM providers are expected to continue rolling out new features such as agent modeling, collaboration, more tool access, pre-built features, reflection, and safety barriers, making it a powerful platform for building agents.

Hybrid Expert Agent Architecture

Builders like Bardeen, Imbue, and MultiOn are leveraging expert mixture (MoE) agent frameworks to address the challenge of providing deterministic results for complex workflows. The idea is to break workflows into tasks assigned to specific agents or functions and provide the necessary ‘scaffolding’ for agents, including data, rich toolsets, and interfaces. An overview of the architecture is as follows:

User Interface

User-facing large models enable users to describe tasks and utilize context windows to provide relevant context, such as a few examples. New UI methods combine user context and interactivity, allowing users to guide agents and improve their approaches. This design enables ‘human-in-the-loop’ verification of final outputs. The design achieves a smoother ‘human-computer interaction’ interface to provide input and verify final outputs.

Task Agents

Workflows can be broken down into different tasks executed by LLM agents, some of which will be detailed below. This optimizes current LLM capabilities and can flexibly utilize task agents to achieve specific functions, providing clean abstractions and options for future upgrades or refactoring.

· Today’s planning agents can propose plans, breaking down user intent into task lists, which are scheduled for execution after human approval. This remains an active area of research.

· Routing tasks are responsible for mapping tasks to suitable AI/ML agents or predefined workflows.

· Functional agents are trained for specific tasks (generative AI or predictive ML models).

· Code generation agents convert tasks into the code required for specific tasks, such as SQL queries.

· Reflective LLMs evaluate quality through iterative outputs and optimize final results. Platforms like Devin have proven this technique effective in improving output accuracy.

Deterministic Runtime

To provide consistently correct final outputs, combining different outputs from different tasks at deterministic runtime has proven to be a good practice. For example, in financial use cases, code generation LLMs generate SQL queries executed at runtime for precise data extraction.

The core design principle of MoE architecture is to use AI/ML models only when needed and utilize predefined workflows/scripts. LLMs are used in design and integrated at deterministic runtime.

AI Interface Between Agents and Humans

As mentioned earlier, the human-computer interaction interface is a key point in today’s architecture. Developers are embedding agents into user contexts in various ways, from inputting in context windows to designing agents as browser plugins that observe user behavior and capture context.

LLM plugins introduce external data or tools, which are key to empowering agents with more skills. Finally, agents can communicate with user platforms like email, productivity, and communication tools via APIs, simulating human workflows.

The interaction interface between agents is an active area of research and development. In MoE models, task agents with different capabilities need to interact as described above. Over time, we can envision agents completing tasks through interaction—similar to the APIs that connect applications today, extending to support AI versions.

Considerations for Enterprises Deploying Automation

1. Most enterprises are already using a range of automation platforms, from classic RPA and task automation platforms for specific tasks to self-developed solutions. The productivity brought by AI remains hype rather than reality. Candidates for GenAI-based automation need clear cost/benefit analyses, as they will follow a maturity curve similar to previous methods.

2. The performance of agents is directly related to the quality and relevance of their training data. For many enterprises, building clean and centralized datasets and data pipelines to train models is a top priority.

3. With the upcoming release of GPT5/Llama3, the landscape of LLMs is rapidly evolving, which will refresh the standards of SOTA. Meanwhile, several models with GPT4-level performance are now offered at very attractive costs. Enterprises now have models from different sources and varying cost-performance levels to choose from based on use cases and functional requirements.

4. At the platform level, there are multiple options in the market. Existing enterprises are embedding AI or providing auxiliary tools to accelerate user time to value. Startups/scaled companies and large model providers are adopting AI-native approaches to reinvent vertical use cases or create new platforms to change costs, performance, and user experience. Workflow and performance benchmarks should drive selection.

5. Today’s large models are very sensitive to prompts, and slight changes can lead to deviations in model outputs. Establishing clear performance metrics at the use case level (as opposed to the model level) is crucial. Governance and data security are equally important. Human-computer interaction is a fundamental feature of all AI deployments today.

Considerations for Automation Builders

1. Builders can adopt a ‘crawl, walk, run’ approach using GenAI in automation platforms. Understanding users, use cases, their performance benchmarks, and leveraging LLMs as tools to match their capabilities with tasks is crucial for building differentiated solutions.

2. Large models are primarily System 1 thinkers (making intuitive responses based on training data). Developers need to differentiate functionalities and predefined functions/scripts, using ML models whenever possible. Focused and targeted datasets are essential for the foundation of models.

3. For complex use cases—continuous experimentation and the right ‘scaffolding’ to integrate user context, access external tools and datasets, reflection mechanisms, etc.—are fundamental aspects of the ‘expert mixture’ agent architecture.

4. Simple text-based UIs are a good first step. Builders innovate by adding real-time interactivity and multi-modal UIs to create more engaging experiences for users, allowing them to track large model task lists, evaluate outputs, and provide proactive feedback to guide outputs.

5. Introducing differentiated datasets with the right governance, considering safety trade-offs, safety barriers, and performance, is crucial to avoid regulatory and compliance issues when deploying in end-user environments.

/ 05 /

Generative AI Agent Use Cases

In our conversations with enterprises, various efforts are underway in agent automation. Below are some detailed use cases:

· The Chief Data Officer of an F100 telecom company: “We are building intelligent workflows that link forms and databases, integrating multiple data sources, and then taking action or making suggestions based on the data.”

· The VP of Data and AI at a global consulting firm is building intelligent workflows that enable data analysts to gain insights from different spreadsheets.

· The Senior VP of Data and AI at an F500 construction and real estate company is building a collection of intelligent programs that combine Palantir, OpenAI, and internal assistants to select winning RFP bids from thousands of submitted RFP bids.

· The Senior VP of a large bank: “I see two active use cases for GenAI. First, the coding assistance tool rolled out to all our engineers, where we have seen productivity increase by over 20% for senior engineers. We look forward to new features here. Second, discussing documents with large models, RAG significantly improves the way we protect privacy while deploying models. Chatbots have been experimental releases, and we are still refining this use case to consider security and compliance.”

· The Chief Digital Officer of a large bank: “Automation will take many forms in our industry. We have a lot of RPA, ITSM vertical automation platforms, etc., and we are building agents on large models. We are actively experimenting with agent automation architectures and continuously learning. Some existing vendors have shown great agility in integrating AI.”

/ 06 /

AI Automation Market Map

The pace of innovation in AI will only continue to accelerate. It is important to note that many of the approaches discussed here are in experimental and early production stages.

As the world of agents and automation evolves, we are committed to actively tracking and updating the developments in this field. At the same time, we welcome opportunities to collaborate with founders to build agent automation, reimagine vertical applications, and differentiate infrastructure platforms, and we look forward to feedback and dialogue with the community.

Written by Xiao Bai

PS:If you have unique insights into the field of AI large models, feel free to scan the QR code to join our large model discussion group.