Multi-Agent Systems: When One Agent Isn't Enough (2026 Guide)

When One Agent Fails (and Splitting Helps)

Symptom	Why it happens	Multi-agent fix
Model calls the wrong tool among many	Tool-selection accuracy degrades past ~10 tools	Router sends the task to a specialist with 3–5 tools
Context mixes unrelated skills	One transcript carries research + formatting + code	Each agent gets a clean, focused context
Latency from a big model doing everything	Every step pays frontier-model cost	Small cheap model routes; big model only where needed
Task has parallel independent parts	One agent is sequential by nature	Run specialists in parallel, merge results

And the honest inverse: for linear tasks, multi-agent is pure overhead. More calls, more latency, more failure points. The decision rule is measurement: if the single agent fails eval (see evaluation patterns) on tasks that a split structure would fix, split.

The Three Patterns

1. Supervisor (Central Router)

One supervisor agent reads the request and routes to specialist agents — research, retrieval, code, formatting. Specialists return results; the supervisor composes the final answer. Best when tasks are clearly classifiable. LangGraph's create_supervisor helper builds this in ~10 lines.

2. Coordinator / Swarm (Peer Handoffs)

Agents delegate to each other — agent A finishes and hands the baton to agent B, which is right for the next step. OpenAI's Swarm popularized the pattern; no central brain, flexible chains, but more routing freedom = more loop risk.

3. Hierarchical (Teams of Agents)

Supervisors supervise sub-supervisors, each managing their own team. Overkill for most products, but the right shape for genuinely large problem spaces — e.g. a legal workflow with a contracts team, a research team, and a filings team.

Building a Local Supervisor in LangGraph

from langchain_ollama import ChatOllama
from langgraph.prebuilt import create_react_agent
from langgraph_supervisor import create_supervisor

llm = ChatOllama(model="qwen3:8b")          # router + workers

researcher = create_react_agent(llm, tools=[search_web_local])
retriever  = create_react_agent(llm, tools=[query_kb])   # ChromaDB
writer     = create_react_agent(llm, tools=[format_doc])

supervisor = create_supervisor(
    [researcher, retriever, writer],
    model=llm,
    prompt=("Route to researcher for web facts, retriever for the "
            "knowledge base, writer for final output."),
)

result = supervisor.invoke({"messages": [
    ("user", "Summarize the KB docs on GDPR and draft a client memo.")
]})

Note the pattern: each agent gets a focused toolset — retriever only touches the vector store — which is the entire point of splitting. The routing prompt is your control surface: the more explicit the routing rules, the fewer misroutes. (langgraph-supervisor is the current package name in 2026; see the LangGraph docs for the latest import path.)

Running It Locally: The Budget Reality

Setup	What works
16GB RAM laptop	1 router + 2 specialists at 4–8B (Qwen3.5-4B router, Qwen3-8B specialists)
24GB unified / 16GB VRAM	Router at 4B + 3–4 specialists at 8–12B — the sweet spot
Server with 2× GPUs	Multiple 12–32B specialists, parallel workstreams

Cost management rule: route with the smallest model that routes correctly, and reserve the big model for the step that needs it. Every agent call is a model call — a careless supervisor burns 3x the tokens for the same answer. Speed optimizations (quantization, KV cache, speculative decoding) apply to every agent in the system.

The Failure Modes (They're Specific)

Routing loops. A delegates to B, B delegates back to A. Fix: max-hops counter in the graph (LangGraph's recursion_limit), and supervisors instead of peer handoffs for risky chains.
Context pollution. Every agent re-reading the full transcript bloats cost and confuses focus. Fix: give each agent only its slice of state.
Cascading errors. One agent's hallucinated output becomes the next agent's input. Fix: validate outputs at handoff boundaries (schema checks, faithfulness spot-checks).
Cost explosion. N agents × M steps × tokens. Fix: measure per-step cost; the supervisor prompt is the cheapest lever.

These are exactly why the single-agent tutorial comes first — you need the baseline to justify the split, and the graph machinery to control it.

Frequently Asked Questions (FAQ)

What is a multi-agent system?

A multi-agent system is multiple AI agents collaborating on one task — each specialized (research, coding, retrieval, QA), coordinated by a router or supervisor that decides which agent handles what and when. The structure mirrors how human teams divide work.

When should I use multiple agents instead of one?

When the task spans distinct skills that fight each other in one context (long research + strict formatting), when tools exceed ~10 (models degrade), when you need parallel workstreams, or when different steps need different models (a small fast one for routing, a big one for deep reasoning).

When should I NOT use multiple agents?

For simple linear tasks. Multi-agent adds latency (more model calls), cost, and failure points — a confused supervisor routes badly, and failures cascade. Rule of thumb: start single-agent, split only when the single agent measurably fails or the task genuinely parallelizes.

What are the main multi-agent patterns?

Three dominate: supervisor (a central agent routes to specialists), coordinator/handoff (agents delegate tasks to each other peer-to-peer, the "swarm" pattern), and hierarchical (teams of agents with sub-supervisors — for very large problems). LangGraph supports all three.

Can multi-agent systems run locally?

Yes — a supervisor routing between 3–5 specialist agents, each on a local model via Ollama, is very feasible on 16–32GB machines. Costs scale with the number of calls, so keep the routing cheap (small model) and the heavy work targeted.

What are the failure modes of multi-agent systems?

Routing loops (agent A delegates back to B forever), context pollution (each agent re-reads the whole transcript), cost explosion, and cascading errors where one agent's mistake poisons the next. Mitigations: max-hops, shared but structured state, per-agent isolation, and evaluation.