The mental model: fixing a light bulb vs. renovating an apartment
Imagine you need to fix a light bulb. That's a single, well-scoped task, so you call an electrician. You hand them the task, they bring their tools and expertise, and the job gets done. That's a single agent: one LLM instance with tools, a system prompt, and a conversation history, all operating in one context window and one decision loop.
Now imagine you want to renovate an entire apartment. That's not one task, it's a goal made up of many specialized tasks: electrical work, plumbing, woodwork, gardening. Unless you have a do-it-all handyman, you delegate to different experts, each with their own tools and area of expertise. That's a multi-agent system.
Throughout this post, the diagrams share one visual language: amber is a coordinator/router making a decision, blue is a specialist agent doing work, purple is shared state, green is a start/success terminal, and red flags a cost or trade-off.
Multi-agent systems are fundamentally about dividing cognitive labor. Just as organizations divide work among specialists, multi-agent systems divide AI work among focused agents that excel in their own domain. Some tasks run in parallel (the electrician and gardener don't need to wait on each other), while others have hard sequential constraints (you might need plumbing finished before electrical work starts).
Anatomy of a multi-agent system
Every multi-agent system is made up of four core components:
- Agents - Independent units with their own LLM, prompt, tools, and state access. Key design decisions: how specialized should each agent be, what tools does it need, and what are its personality and constraints (its scope).
- Coordination mechanism - Defines how agents work together: who decides routing, how agents communicate, and what triggers a transition from one agent to another.
- State management - Governs data flow across agents: what stays persistent, what's shared vs. isolated, and how context flows through and between agents.
- Entry and exit points - The interface with the user: which agent(s) receive input, and which agent(s) get to respond.
When multi-agent systems make sense
Don't reach for a multi-agent system just because it's available, use one when the signals actually point to it:
- Tool overload - Once a single agent creeps past ~30 tools, the LLM starts making poor or confused tool choices. Distribute tools across specialists instead (e.g., 30 tools → 3 specialists with ~10 each).
- Context window limits - Long-running conversations degrade quality as context grows. Splitting work across agents enables context isolation, where each agent only holds what it needs.
- Sequential constraints - When a step must be verified or completed before the next can start (e.g., verify identity before granting account access), a multi-agent handoff can architecturally enforce that agent B is unreachable until agent A finishes.
- Strong parallelization needs - If subtasks are truly independent (the gardener doesn't need to wait on the electrician), dispatch them to specialists simultaneously instead of forcing a sequential flow.
- Organizational boundaries - If your system mirrors an org chart (sales, support, engineering), model it the same way, one agent (or team) per domain, composed through a central layer. This is the same heuristic that justifies microservices over a monolith.
If your system is still simple (under ~15 tools), has tight latency or budget constraints, or is early in development, stick with a single agent.
Benefits vs. trade-offs
Benefits:
- Better accuracy - specialists reliably outperform generalists in their own domain.
- Better tool selection - fewer, more relevant tools per agent means better decisions.
- Maintainability - agents can be tested, evaluated, and reasoned about independently; no single point of failure.
- Parallel execution - independent subtasks run simultaneously instead of paying an avoidable latency cost.
- Graceful expansion - new specialist agents can be added (or removed) without destabilizing the rest of the system.
- Enforced compliance - business rules become structural (agent A architecturally cannot see agent B's data) instead of just advisory prompt instructions.
Trade-offs:
- Increased latency - every agent transition is another LLM call.
- Higher cost - handoffs and coordination mean repeated calls and more tokens spent.
- More complexity - more contracts between agents, more failure modes, harder debugging.
- Conversation coherence - handoffs must be pristine; if an agent lacks context, it hallucinates or goes off-track.
Multi-agent concepts in LangGraph
The LangGraph primitives you already know map directly onto multi-agent concepts:
| Concept | Represents |
|---|---|
| Nodes | Agents, sub-agents, functions, or a single computation (e.g., a routing/classification step) |
| Edges | Transitions between agents; direct or conditional (conditional edges enable runtime routing) |
| State | Information flow between agents |
| Checkpointer | Persistence for long-running, resumable, multi-turn workflows (flows down to all sub-agents) |
| Command | Update state within an agent and/or navigate to another agent |
| Send | Dispatch operations to run in parallel |
Design patterns for multi-agent systems
1. Sub-agents
A main agent acts as a supervisor/coordinator, delegating tasks to specialist sub-agents and collecting their results before deciding what happens next. This is the closest pattern to the renovation analogy: you're the coordinator, calling in specialists as needed and staying in the loop between each one.
Every arrow returns to the supervisor, it stays in control of sequencing, retries, and what to tell the user.
2. Handoffs
A strictly sequential, peer-to-peer flow. One agent finishes its work and hands off directly to the next, carrying the accumulated context forward. There's no central coordinator: control passes along the chain, and each agent is unreachable until the previous one completes.
Because there's no coordinator to fall back on, the handoff itself has to be complete, if agent B drops context agent A had, agent C never recovers it.
3. Skills
A single, "shape-shifting" agent rather than a team. It stays one agent, but loads different skills or tool sets on demand based on what the task requires, self-augmenting instead of delegating to a separate specialist.
One identity, one context window, but a growing toolbox it reaches into only as needed.
4. Router
Takes a goal, breaks it into independent pieces, dispatches them to specialists in parallel (by design, not as a conditional branch), then synthesizes their results into one answer.
This is the pattern behind Send in LangGraph: dispatch N branches at once, then fan them back in.
And when a task doesn't cleanly fit any of these, you can always build a custom flow, or hybridize patterns, since LangGraph gives you full control over how the graph is wired.
Each of these patterns will get its own deep dive in upcoming posts, this is the map before we walk the territory.