02. AI Agent vs Large Language Model

An LLM is a reasoning engine. An AI Agent is an autonomous worker. The LLM thinks; the Agent acts.

This is one of the most important distinctions in modern AI. Many people say “ChatGPT is an agent” or “Claude is an agent” — but they’re not. LLMs and Agents are fundamentally different, and understanding the difference is essential for building effective AI systems.

flowchart LR
    subgraph LLM["LLM (Teacher)"]
        IN1["Question"] --> BRAIN1["Knowledge & Reasoning"] --> OUT1["Answer"]
    end

    subgraph AGENT["AI Agent (Personal Assistant)"]
        IN2["Goal"] --> BRAIN2["LLM Reasoning"]
        BRAIN2 --> PLAN2["Planning"]
        PLAN2 --> TOOL["Tool Use"]
        TOOL --> RESULT["Action Result"]
        RESULT --> BRAIN2
        MEM2["Memory"] -.-> BRAIN2
    end

    style LLM fill:#3b82f6,color:#fff
    style AGENT fill:#22c55e,color:#fff

When you ask ChatGPT to “book a flight,” it happily writes a step-by-step guide. But it doesn’t actually book the flight. The user is left with instructions, not results.

This confusion leads to:

• Wrong architecture decisions — Building agent systems when simple LLM calls would suffice
• Over-engineering — Adding tools, memory, and planning when a single prompt is enough
• Under-engineering — Expecting an LLM to act like an agent without the infrastructure

LLMs and Agents exist on a spectrum. Simple tasks need LLMs. Complex tasks need Agents.

flowchart TD
    Q["What do you need the AI to do?"]
    Q -->|"Answer a question"| SIMPLE["Use an LLM\n(GPT-4o, Claude)"]
    Q -->|"Write content"| SIMPLE
    Q -->|"Summarize a document"| SIMPLE
    Q -->|"Translate text"| SIMPLE

    Q -->|"Research a topic across 10 websites"| AGENT["Use an AI Agent\n(Search + Read + Synthesize)"]
    Q -->|"Build and deploy an app"| AGENT
    Q -->|"Automate a multi-step workflow"| AGENT
    Q -->|"Interact with external systems"| AGENT

    style SIMPLE fill:#3b82f6,color:#fff
    style AGENT fill:#22c55e,color:#fff

An LLM is a teacher. You ask a question, they give you an answer. They’re incredibly knowledgeable, but they don’t do anything beyond answering. They don’t search the internet, they don’t run experiments, they don’t call anyone on your behalf.

An AI Agent is a personal assistant. You give them a goal: “Plan my team’s offsite.” They:

1. Check everyone’s calendar (tool: calendar API)
2. Search for venues (tool: web browser)
3. Compare prices (tool: calculator/spreadsheet)
4. Send emails to shortlisted venues (tool: email)
5. Draft an itinerary (tool: document editor)
6. Present the completed plan to you

The teacher (LLM) tells you how to plan an offsite. The assistant (Agent) actually plans it.

flowchart LR
    subgraph LLM_ARCH["LLM Architecture"]
        direction TB
        P1["Prompt"] --> M1["Pretrained LLM"]
        M1 --> R1["Response"]
    end

    subgraph AGENT_ARCH["Agent Architecture"]
        direction TB
        G1["Goal"] --> O1["Orchestrator"]
        O1 --> M2["LLM (Reasoning)"]
        M2 --> T1["Tool 1: Search"]
        M2 --> T2["Tool 2: Code"]
        M2 --> T3["Tool 3: Files"]
        T1 --> O1
        T2 --> O1
        T3 --> O1
        O1 --> O2["Observer"]
        O2 --> O3["Reflection"]
        O3 -->|"Continue"| O1
        O3 -->|"Done"| AR1["✅ Result"]
        MEM1["Memory Store"] -.-> O1
    end

    style LLM_ARCH fill:#3b82f6,color:#fff
    style AGENT_ARCH fill:#22c55e,color:#fff

flowchart TD
    TASK["What's the task?"]
    TASK --> CHECK1["Does it require\nmultiple steps?"]
    CHECK1 -->|"No, single response"| LLM_USE["Use an LLM\n✅ Simpler, cheaper, faster"]
    CHECK1 -->|"Yes, multiple steps"| CHECK2

    CHECK2["Does it need\nexternal tools or data?"]
    CHECK2 -->|"No, just reasoning"| LLM_USE
    CHECK2 -->|"Yes, needs tools"| CHECK3

    CHECK3["Does it need\nmemory across steps?"]
    CHECK3 -->|"No"| SIMPLE_AGENT["Use a simple Agent\n(LLM + tool calls)"]
    CHECK3 -->|"Yes"| FULL_AGENT["Use a full Agent\n(LLM + tools + memory + planning)"]

    style LLM_USE fill:#3b82f6,color:#fff
    style SIMPLE_AGENT fill:#f59e0b,color:#fff
    style FULL_AGENT fill:#22c55e,color:#fff

The most important insight: Agents don’t replace LLMs — they use them.

sequenceDiagram
    participant Goal as User Goal
    participant Agent as AI Agent
    participant LLM as LLM (Brain)
    participant Tool as Tool/API
    participant Env as Environment

    Agent->>LLM: "I need to book a flight. What's the first step?"
    LLM-->>Agent: "First, search for available flights on the travel API"

    Agent->>Tool: Call travel API (origin, destination, dates)
    Tool->>Env: Query flight database
    Env-->>Tool: Flight results
    Tool-->>Agent: 15 flights found

    Agent->>LLM: "Here are 15 flights. Which one should I book? Budget: $500 max"
    LLM-->>Agent: "Flight #3 is under budget, best departure time, and has the shortest layover"

    Agent->>Tool: Book flight #3
    Tool->>Env: Process booking
    Env-->>Tool: Booking confirmed (PNR: ABC123)
    Tool-->>Agent: Confirmation received

    Agent->>LLM: "Flight booked. What should I do next?"
    LLM-->>Agent: "Task complete. Send confirmation to user."

    Agent->>Goal: "✅ Flight booked! PNR: ABC123"

The LLM provides reasoning at each step. The Agent provides the orchestration, tool access, and persistence.

Q: Is ChatGPT an AI Agent? Why or why not?

No. ChatGPT is a Large Language Model, not an AI Agent. ChatGPT can answer questions and generate text, but it cannot use tools, take actions in the world, maintain persistent memory, or work autonomously toward a goal. It processes one prompt at a time and then stops.

Q: Can an LLM exist without an Agent?

Yes! An LLM is a standalone model that generates text. Many applications use LLMs directly — such as chatbots, content generators, and translators. An Agent is an optional layer on top of an LLM.

Q: Explain how an LLM and an Agent work together in a production system.

The Agent is the orchestrator, and the LLM is the reasoning engine. The Agent receives a goal, asks the LLM for a plan, executes the plan using tools, collects results, asks the LLM to evaluate the results, and repeats until the goal is achieved. The LLM never directly interacts with tools or the environment — that’s the Agent’s job.

Q: Design an architecture where the same LLM powers both a simple chatbot and a complex research agent. How would you structure this?

Use a router pattern. Single LLM endpoint, but two orchestration layers above it: (1) Simple path — User query goes directly to LLM, response returned immediately. Cheap and fast. (2) Agent path — User query goes to an agent orchestrator that uses the same LLM for reasoning, planning, tool selection, and reflection. The router decides which path based on query complexity (simple questions → direct, multi-step research → agent). This gives you the best of both worlds.

Q: How would you measure the value added by turning an LLM into an Agent?

Measure three metrics before and after: (1) Task completion rate — What % of tasks get fully completed? (2) User time saved — How much manual work does the agent eliminate? (3) Cost per completed task — Total tokens + API calls + tool costs. A good agent should increase task completion by 3x+, save users 80%+ of their time, while keeping cost under $0.50 per task. If the agent doesn’t improve these metrics, the LLM alone was sufficient.

Q: Design a system that uses both LLM and Agent patterns for a customer support platform.

Tier 1 (LLM) — Simple FAQs: “What’s your return policy?” → Direct LLM response. No agent needed. Tier 2 (Simple Agent) — Order status: Agent calls order lookup API, LLM interprets results, responds to customer. Tier 3 (Full Agent) — Complex issues: Agent researches customer history, checks multiple systems, drafts resolution, gets human approval, executes. Each tier handles its complexity level. Routing based on query intent classification.

Previous: 01 — What is an AI Agent?

Next: 03 — Agent Lifecycle