When to Use AI Agents: A Simple Flowchart

When and how to use AI Agents vs. AI Workflows for your use cases

Key Points

• Use agentic AI for tasks needing flexibility and dynamic decision-making, like coding assistants or virtual assistants.

• Use AI workflows for well-defined, repetitive tasks with clear steps, such as order processing or customer support responses.

• Start with a simple LLM call if the task is a basic query; escalate to workflows or agents as needed.

Agentic AI vs. AI Workflows - The 30 Second Version

What Are They?

Agentic AI, or AI agents, are systems that autonomously plan and adapt, making decisions on the fly. They're great for complex, open-ended tasks. AI workflows, on the other hand, follow predefined steps, ideal for consistent, repetitive processes.

How to Choose?

First, check if a simple LLM call with retrieval can handle your task, like answering a quick question. If not, decide: Is your task well-defined with clear steps? If yes, use workflows; if no, and it needs flexibility, go for agentic AI. This approach keeps things simple and effective.

Cost and Latency

Agentic AI can be more expensive and slower due to its dynamic nature, so for budget-sensitive projects, workflows might be better even for complex tasks, as long as steps are clear.

Agentic AI vs. AI Workflows - In Detail

The paragraph above was obviously a gross simplification. But it should give you an idea about the fundamental differences between the two approaches and what the factors are that your decision will depend on.

Let’s go over a more comprehensive exploration of when to use agentic AI (AI agents) versus AI workflows, drawing heavily on Anthropic's classification system as a starting reference. As such, the remainder of this article is supposed to guide decision-making, offering both theoretical insights and practical applications, with a focus on building effective AI systems.

Background and Definitions

To begin, we need to clarify the terminology we’re building on. Anthropic defines two key categories relevant to building effective AI systems:

• AI Workflows: These are systems where large language models (LLMs) and tools are orchestrated through predefined code paths. They are designed for tasks with clear, predictable steps, ensuring consistency and ease of debugging. Examples include automating order processing or managing customer accounts, where the sequence of actions is well-established.

• Agentic AI (AI Agents): These, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how tasks are accomplished. They are suited for open-ended, complex tasks requiring adaptability, such as autonomous coding assistants or virtual assistants handling varied user requests.

This distinction is crucial, and it is recommend to start with the simplest solution possible while only increasing complexity when necessary. It’s very rarely the case that AI Agents should be your first choice for solving a problem.

Decision-Making Framework

The decision process for choosing between agentic AI and AI workflows can be broken down into a structured flowchart, which we will describe in detail, followed by simple guidelines. This, of course, is a work in progress, as the current generation of AI Agents is still in the process of emerging and it’s unclear when this rapid progress will come to a halt. We’ve seen compute costs drop massively within the two years, after all.

Flowchart Description

The decision process can be visualized as follows, with each step representing a binary choice leading to the appropriate system:

1. Start: Begin by assessing the task at hand.

2. Decision 1: Can the task be handled with a single LLM call with retrieval?

   • Yes: Proceed to use a single LLM call with retrieval, which is sufficient for simple queries like answering factual questions or generating basic text. This approach leverages retrieval mechanisms to provide context, keeping costs and complexity low.

   • No: If the task cannot be handled by a single call, move to the next decision.

3. Decision 2: Is the task well-defined with clear, predefined steps?

   • Yes: Use AI workflows. These are ideal for tasks that can be broken down into a sequence of predefined steps, ensuring predictability and consistency. Examples include automated customer support responses or data processing pipelines.

   • No: Use agentic AI. These are necessary for tasks that are open-ended or require dynamic decision-making, such as coding assistants solving GitHub issues or virtual assistants planning and executing varied tasks.

This basic flowchart reflects the principle of starting with the simplest solution (single LLM call) and escalating to workflows for well-defined tasks, and finally to agents for more complex, adaptive needs.

Guidelines for Implementation

To provide practical guidance, we can categorize the use cases and considerations for each option:

• Single LLM Call with Retrieval:

  • Use Case: Suitable for simple queries where the LLM can provide a direct answer, possibly enhanced by retrieving relevant information from a database or knowledge base. For instance, answering a customer's question about product specifications.

  • Advantages: Low cost, low latency, and minimal setup. It aligns with Anthropic's recommendation to optimize single LLM calls with retrieval and in-context examples for many applications.

  • Example: A chatbot answering, "What are the opening hours of our store?" by retrieving the information and responding directly.

• AI Workflows:

  • Use Case: Best for tasks that are well-defined and repetitive, with clear steps that do not require the LLM to make decisions about which step to take next. Examples include order fulfillment processes, from receiving an order to shipping, or managing customer accounts.

  • Advantages: Predictability, consistency, and ease of debugging. Workflows follow predefined code paths, making them suitable for tasks like routing customer inquiries to different departments based on categories (e.g., billing, technical support).

  • Considerations: Ensure inputs and outputs for each step are well-defined to avoid errors. Often implemented via patterns like prompt chaining and routing as part of workflows, which can be implemented in a few dozen lines of code without complex frameworks.

  • Example: A banking AI chatbot that processes balance checks by following a predefined sequence: verify user, retrieve account data, format response.

• Agentic AI:

  • Use Case: Ideal for tasks that are complex, open-ended, or require flexibility and model-driven decision-making. These include scenarios where the LLM needs to plan independently, use external tools, or adapt to changing conditions. Examples include coding agents solving GitHub issues, as seen in benchmarks like SWE-bench Verified, or virtual assistants handling varied user requests like scheduling meetings and generating reports.

  • Advantages: High adaptability, suitable for dynamic environments. Agents can break down problems into manageable steps, collaborate with other agents, and use tools like web searches or API calls, as noted in the blog post.

  • Considerations: Higher costs and latency due to dynamic processing. It’s recommended to test in sandboxed environments and to prioritize transparency by showing planning steps. Another important factor for agents utilizing complex tools (e.g., Azure CLI, GitHub) is to build a thorough agent-computer interface (ACI) through tool documentation and testing. Your agents will only be able to use the right tools for your tasks if they have access to sufficient information about them.

  • Example: An AI agent that autonomously writes and tests code to fix a GitHub issue, verified by automated tests and human review, as part of the SWE-bench initiative.

Additional Considerations

While the flowchart and guidelines provide a clear path, they are only a starting point. The main goal of this article is to raise awareness about the fact that “throwing AI Agents at everything” is not a viable strategy - and to provide a starting point for decision making.

A lot of the additional factors to consider, particularly around cost and performance, are trade-offs specific to your individual use cases. Agentic systems often trade latency and cost for better task performance, making them less suitable for budget-sensitive projects where workflows can suffice. For instance, if a task is well-defined but very complex, workflows might still be preferred over agents to maintain efficiency, as long as the steps can be adequately predefined.

Customer service is a great example: it often requires close-to-realtime latency and strong guardrails since it’s a customer facing application (and a lot of the customers contacting customer service are already not in the best mood). While AI Agents promise to do a much better job than previous technologies, the bar for reliability and safety is high. And you can build pretty advanced chatbots with workflows already. In fact, most chatbots currently running in production have little to do with agentic AI (even if they claim otherwise). Things like tools for customer data and order history are much more important and can be implemented as workflows for standard queries. A pragmatic compromise could be to utilize agents for more ambiguous requests. The two design approaches aren’t mutually exclusive - which is another important consideration. Use whatever works best for your use case.

Conclusion

In conclusion, the decision between agentic AI and AI workflows hinges on the task's nature: use single LLM calls for simple queries, workflows for well-defined, repetitive tasks, and agents for open-ended, dynamic tasks requiring adaptability. This approach aligns with recommendations for simplicity, transparency, and thorough testing, ensuring effective AI system design. The flowchart and guidelines provided above offer a practical starting point for implementation, supported by real-world examples and publicly shared learnings from leading AI companies like Anthropic & HuggingFace.

---