Multi-Agent Systems and Agent Architectures

A Multi-Agent System (MAS) is a system where multiple agents interact or work together to solve problems, make decisions, or complete tasks.

Instead of relying on a single agent, the work can be distributed among agents with different roles, skills, knowledge, or tools.

Multi-agent systems can include traditional autonomous agents such as robots and software agents, as well as modern LLM-powered agents that collaborate on complex tasks.

In this chapter, we will explore multi-agent systems, agent architectures, coordination patterns, and LLM-based multi-agent architectures.

What Is a Multi-Agent System?

A practical definition is:

A multi-agent system is a system in which two or more agents interact while working on tasks or goals within a shared problem space or environment.

An agent is a software or physical entity that can observe information, make decisions, and perform actions with some degree of autonomy.

Example: Warehouse Robots

Imagine a warehouse containing 20 autonomous robots.

Each robot can:

• Detect its current position

• Receive delivery tasks

• Move through the warehouse

• Avoid obstacles

• Communicate with other robots

• Decide which route to take

                Warehouse Environment
                        │
        ┌───────────────┼───────────────┐
        ↓               ↓               ↓
    Robot 1         Robot 2         Robot 3
        ↔               ↔               ↔
     Communication / Coordination

The warehouse is a multi-agent system because multiple autonomous agents operate and interact within the same environment.

Traditional agent systems commonly emphasize characteristics such as autonomy, responsiveness to the environment, proactive behavior, and social interaction. An agent architecture defines how capabilities such as perception, decision-making, communication, and action are organized within an agent.

In an LLM-based multi-agent architecture, the agents may instead be specialized software components such as:

Research Agent → Data Agent → Writer Agent → Reviewer Agent

Each agent handles a different responsibility while contributing to the larger task. This is a common pattern in modern multi-agent system architecture in AI.

Single-Agent vs Multi-Agent Systems

Understanding single-agent vs multi-agent systems helps you decide whether a task can be handled by one agent or benefits from multiple specialized agents.

Why Use Multiple AI Agents?

Multiple AI agents are useful when dividing a complex problem into specialized responsibilities provides a real engineering advantage.

Consider a financial research assistant. Instead of giving one agent instructions and tools for financial data, news, valuation, risk analysis, and report writing, the system could divide the work:

                User Request
                     ↓
               Supervisor Agent
                     ↓
        ┌────────────┼────────────┐
        ↓            ↓            ↓
 Financial Data    News       Valuation
     Agent          Agent        Agent
        \            |            /
         └───────────┼───────────┘
                     ↓
                 Risk Agent
                     ↓
                Report Agent
                     ↓
                Final Response

This approach can provide several benefits:

• Specialization — Each agent focuses on a specific responsibility.

• Context isolation — Agents receive only the information relevant to their tasks.

• Parallel execution — Independent tasks can sometimes run simultaneously.

• Clear boundaries — Different agents can own different tools or parts of a workflow.

• Modularity — Individual agents can be developed and improved separately.

However, more agents are not automatically better.

A multi-agent architecture can introduce more model calls, higher cost, additional latency, more state, additional failure paths, and harder debugging.

Therefore, use multiple agents when the benefits of specialization, parallelism, or separation of responsibilities justify the additional coordination complexity.

Multi-Agent System Architecture

A multi-agent system architecture defines how multiple agents are organized and how they communicate, share state, use tools, and coordinate their work.

A general architecture may look like this:

                    User / Environment
                           ↓
                  Orchestrator / Router
                           ↓
              ┌────────────┼────────────┐
              ↓            ↓            ↓
           Agent A       Agent B      Agent C
              ↕            ↕            ↕
         ┌──────── Shared State ────────┐
         │                              │
         └──── Communication Layer ─────┘
              ↓            ↓            ↓
           Tools         Search       External
           / APIs       / Database     Systems
              \            |            /
               └───────────┼───────────┘
                           ↓
                     Final Result

A good multi-agent system architecture design should answer a few important questions:

Common multi-agent architecture patterns include supervisor-worker, hierarchical, decentralized, router-based, planner-executor, agent handoffs, and graph-based workflows.

There is no single architecture that works best for every application. The right design depends on the task complexity, coordination requirements, workflow predictability, and level of agent autonomy.

Specialized Agents

A specialized agent focuses on a specific responsibility instead of trying to handle every type of task.

For example, an e-commerce system might use:

• Catalog Agent — Handles product information.

• Order Agent — Checks order status and details.

• Returns Agent — Handles return and refund policies.

• Payment Agent — Handles payment-related queries.

Each agent receives only the instructions, tools, and context relevant to its role.

Specialization can improve tool selection and context management. Instead of giving one general agent dozens of unrelated tools, each agent works with a smaller and more relevant set.

However, agent responsibilities must have clear boundaries. Poorly defined roles can cause duplicated work, incorrect routing, or agents repeatedly handing tasks to one another.

Collaborative Agents

Specialization defines what each agent is responsible for, while collaboration defines how multiple agents work together toward a result.

For example, several agents could review the same software design:

• Security Agent — Identifies security risks.

• Performance Agent — Evaluates scalability and performance.

• Reliability Agent — Analyzes failures and recovery.

Their results can then be combined:

                    Software Design
                          ↓
              ┌───────────┼───────────┐
              ↓           ↓           ↓
           Security   Performance  Reliability
            Agent        Agent        Agent
              \           |           /
               └──────────┼──────────┘
                          ↓
                     Synthesis
                          ↓
                    Final Review

Agent collaboration can be sequential, parallel, iterative, or competitive, depending on the problem.

In traditional multi-agent systems, agents may also have different or competing goals. In many LLM-based multi-agent applications, however, specialized agents are designed to collaborate toward a shared application or user objective.

Hierarchical Agent Architecture

A hierarchical agent architecture organizes agents into multiple levels, where higher-level agents coordinate or delegate work to lower-level agents.

For example:

Executive Agent
      ↓
Team Supervisors
   ↓          ↓
Workers     Workers

Higher-level agents usually handle planning, coordination, and delegation, while lower-level agents perform more specialized tasks.

For example, an enterprise research system might look like:

Research Director
├── Market Research Supervisor
│   ├── Web Researcher
│   └── Competitor Researcher
│
└── Financial Analysis Supervisor
    ├── Financial Statement Agent
    └── Valuation Agent

This architecture is useful when a system has many specialized agents and a single supervisor would become difficult to manage.

However, additional hierarchy can increase communication overhead and latency. Decisions made incorrectly at higher levels can also affect multiple agents below them.

Hierarchical architectures work best when complex tasks can be divided into clear levels of responsibility and delegation.

Supervisor Agent Pattern

The supervisor agent pattern uses a central agent to coordinate specialized agents and decide what should happen next.

User
 ↓
Supervisor Agent
 ├──→ Research Agent
 ├──→ Database Agent
 └──→ Coding Agent
 ↓
Final Response

The supervisor can decide:

• Which agent should handle a task

• What instructions to give that agent

• Whether another agent should run next

• When enough information has been collected

Unlike simple routing, the supervisor can make multiple context-aware decisions throughout the workflow.

This pattern is useful for dynamic workflows where the required sequence of tasks cannot always be predicted in advance.

The main drawback is that the supervisor can become a bottleneck or single point of coordination failure. Poor delegation decisions can send the entire workflow in the wrong direction.

Planner-Executor Pattern

The planner-executor pattern separates planning from task execution.

Suppose the user asks:

"Investigate why API latency increased after today's deployment."

The planner might create:

1. Inspect deployment changes.
2. Compare latency metrics.
3. Check database performance.
4. Inspect external dependencies.
5. Identify and summarize the likely cause.

One or more executor agents then perform these tasks. The planner may also update the plan when new information is discovered.

This pattern is useful for complex, multi-step tasks because it creates an explicit plan and separates high-level reasoning from execution.

However, a poor plan can lead to unnecessary or incorrect execution. For predictable workflows, deterministic workflow logic may be simpler and more reliable than dynamically generating every step with an LLM.

Router Agent Pattern

The router agent pattern determines which specialized agent should handle an incoming request.

For example:

"Where is my package?"
        ↓
   Shipping Agent

"I was charged twice."
        ↓
    Billing Agent

"How do I reset my password?"
        ↓
Technical Support Agent

A router usually makes a routing or classification decision and sends the request to the appropriate agent. Depending on the architecture, it may also route a request to multiple agents.

The key difference between a router and supervisor is the amount of control they have over the workflow:

Use a router when the main problem is deciding who should handle the request. Use a supervisor when the system also needs to decide what should happen next throughout the task.

Agent Roles and Responsibilities

Before writing prompts or code, clearly define what each agent is responsible for.

Each agent should ideally have:

Clear responsibilities prevent duplicated work, conflicting actions, and unclear ownership.

For example:

• Refund Eligibility Agent — Determines whether a customer qualifies for a refund.

• Refund Execution Agent — Issues the refund after approval.

Separating these responsibilities is especially important because reasoning and irreversible actions have different risk levels. The agent making a decision does not always need permission to execute that decision.

Task Delegation

Task delegation in multi-agent systems means assigning a task to the agent best suited to perform it.

Delegation can be static or dynamic.

Static delegation follows predefined rules:

Invoice Question → Accounting Agent

Dynamic delegation chooses an agent at runtime:

User Request
     ↓
Supervisor / Router
     ↓
Choose Specialist

For effective delegation, the receiving agent should get a clear objective and enough relevant context to complete the task.

Instead of:

Research this.

A better task would be:

Find evidence explaining the increase in API latency.
Focus on database and network causes.
Return the three strongest findings with supporting metrics.

A useful principle is:

Delegate the objective and relevant context, not the entire application's history.

This keeps each agent focused while reducing unnecessary context and token usage.

Agent-to-Agent Communication

Agent-to-agent communication is the exchange of information between agents.

Agents may communicate information such as:

• Task requests

• Observations or results

• Status updates

• Errors

• Structured events

Traditional multi-agent systems use formal communication standards such as FIPA ACL (Agent Communication Language), which defines structured messages with fields such as sender, receiver, content, language, and ontology.

Modern LLM-based systems often use simpler structured messages, for example:

{
  "task": "check_return_eligibility",
  "order_id": "A123",
  "reason": "damaged item"
}

Agents may also communicate indirectly through shared application state:

{
  "customer_request": "...",
  "order": {},
  "policy_result": "...",
  "decision": "..."
}

Structured communication is often easier to validate, control, and debug than unrestricted natural-language messages between agents.

Agent Coordination

Agent coordination ensures that multiple agents work together without conflicting with or duplicating each other's actions.

The difference is simple:

• Communication: How does information move between agents?

• Coordination: How do agents organize their actions correctly?

For example, two warehouse robots may communicate their positions but still need coordination to decide which robot enters a narrow aisle first.

In software-based multi-agent systems, common coordination problems include:

• Duplicate work

• Conflicting decisions

• Race conditions

• Circular delegation

• Resource contention

• Inconsistent results

Coordination can be implemented using task ownership, priorities, queues, locks, scheduling, voting, deadlines, or centralized control.

For example:

Only the Payment Agent can execute refunds.

Other agents can recommend a refund,
but they cannot issue one.

This creates a clear responsibility boundary and prevents multiple agents from performing the same sensitive action.

Agent Orchestration

Agent orchestration controls the overall execution of a multi-agent workflow.

An orchestrator can determine:

• Which agent runs first or next

• Which tasks can run in parallel

• When a task should be retried

• When human approval is required

• How failures are handled

• When the workflow should stop

For example:

Request
   ↓
Classify
   ↓
Gather Data
   ↓
Analyze
   ↓
Human Approval
   ↓
Execute
   ↓
Respond

Frameworks such as LangGraph support this type of orchestration using state, nodes, and edges to define how an agent workflow executes.

The difference between communication, coordination, and orchestration can be summarized as:

Shared State and Context

Shared state stores information that multiple agents or workflow steps may need.

For example, an incident-response system might maintain:

{
  "incident": "...",
  "service": "checkout-api",
  "deployment": "v82",
  "metrics": {},
  "logs": [],
  "hypotheses": [],
  "approved_action": null
}

However, every agent does not need access to every piece of information.

For example:

• Log Agent needs the incident details and logs.

• Deployment Agent needs deployment information.

• Incident Commander may only need summarized findings from other agents.

This leads to an important principle:

Shared state does not mean shared context.

The system can maintain a common source of state while giving each agent only the relevant context required for its task.

This improves context efficiency, reduces unnecessary token usage, and helps agents stay focused on their responsibilities.

In graph-based architectures such as LangGraph, nodes can read and update shared state, while edges determine which node or agent executes next.

Challenges in Multi-Agent Systems

Multi-agent systems provide specialization and flexibility, but they also introduce additional complexity.

• Latency — Multiple agent calls can increase response time.

• Cost — More LLM calls mean higher token and compute costs.

• Coordination — Agents may duplicate work, conflict, or delegate tasks incorrectly.

• Debugging — Failures can originate from agents, tools, routing, or shared state.

• State Management — Shared data must be updated and synchronized correctly.

• Reliability — More components create more possible failure points, and errors can propagate between agents.

• Context Management — Too little context causes missing information; too much creates noise and cost.

• Security — Each agent should receive only the tools and permissions required for its role.

Because of these challenges, use a multi-agent architecture only when specialization, parallelism, or separation of responsibilities provides enough value to justify the added complexity.

Real-World Multi-Agent Examples

Multi-agent systems are used in both traditional AI and modern LLM applications.

Common examples include:

• Autonomous robot teams

• Traffic and logistics systems

• Multi-agent research assistants

• Customer-support systems

• Coding agents

• Financial analysis systems

• Incident-response systems

Example: Production Incident Response

Suppose an online retailer experiences a sudden increase in checkout latency after a deployment.

A multi-agent system could use:

• Incident Router — Classifies the incident.

• Deployment Agent — Checks recent deployment changes.

• Observability Agent — Analyzes logs and metrics.

• Dependency Agent — Checks databases and external services.

• Incident Supervisor — Combines findings and recommends the next action.

                Incident
                   ↓
             Incident Router
                   ↓
        ┌──────────┼──────────┐
        ↓          ↓          ↓
   Deployment  Observability Dependency
     Agent        Agent        Agent
        \          |          /
         └─────────┼─────────┘
                   ↓
           Incident Supervisor
                   ↓
          Recommended Action
                   ↓
          Human Approval
          (if required)
                   ↓
                Execute
                   ↓
            Verify Result

The specialist agents can investigate different causes in parallel. The supervisor combines their findings and recommends an action.

For example, if the evidence shows that a new deployment introduced an inefficient database query, the supervisor may recommend a rollback. Because rollback changes production, the system can require human approval before execution.

This architecture combines parallel investigation, specialized agents, centralized coordination, and controlled execution.

Multi-Agent System Architecture with LangGraph

LangGraph is useful when you need explicit control over state, agent execution, and workflow transitions.

The following simplified example models the production incident-response workflow using StateGraph, START, END, and conditional routing.

from typing import Literal
from typing_extensions import TypedDict
from langgraph.graph import StateGraph, START, END

class IncidentState(TypedDict, total=False):
    incident: str
    deployment_finding: str
    observability_finding: str
    dependency_finding: str
    recommendation: str
    approved: bool
    result: str

def deployment_agent(state: IncidentState):
    return {
        "deployment_finding":
            "Latency increased immediately after deployment v82."
    }

def observability_agent(state: IncidentState):
    return {
        "observability_finding":
            "Checkout latency increased from 300 ms to 4 seconds."
    }

def dependency_agent(state: IncidentState):
    return {
        "dependency_finding":
            "Database health is normal, but checkout query duration increased."
    }

def supervisor(state: IncidentState):
    return {
        "recommendation":
            "Evidence points to deployment v82. "
            "Recommend rollback after human approval."
    }

def approval_step(state: IncidentState):
    # Replace with a real human approval mechanism.
    return {"approved": True}

def execute_rollback(state: IncidentState):
    if not state.get("approved"):
        return {"result": "Rollback blocked: approval required."}

    return {"result": "Rollback executed successfully."}

def route_after_approval(
    state: IncidentState,
) -> Literal["execute_rollback", END]:

    if state.get("approved"):
        return "execute_rollback"

    return END

builder = StateGraph(IncidentState)

builder.add_node("deployment_agent", deployment_agent)
builder.add_node("observability_agent", observability_agent)
builder.add_node("dependency_agent", dependency_agent)
builder.add_node("supervisor", supervisor)
builder.add_node("approval", approval_step)
builder.add_node("execute_rollback", execute_rollback)

# Run investigation agents.
builder.add_edge(START, "deployment_agent")
builder.add_edge(START, "observability_agent")
builder.add_edge(START, "dependency_agent")

# Send findings to the supervisor.
builder.add_edge("deployment_agent", "supervisor")
builder.add_edge("observability_agent", "supervisor")
builder.add_edge("dependency_agent", "supervisor")

builder.add_edge("supervisor", "approval")

builder.add_conditional_edges(
    "approval",
    route_after_approval,
)

builder.add_edge("execute_rollback", END)

graph = builder.compile()

result = graph.invoke({
    "incident": "Checkout latency increased after deployment."
})

print(result["recommendation"])
print(result["result"])

How It Works

• IncidentState stores the shared workflow state.

• Three specialized agents investigate the incident.

• Their findings are added to shared state.

• The supervisor creates a recommendation from the findings.

• The approval step represents a human-in-the-loop safety boundary.

• add_conditional_edges() decides whether the rollback can execute.

• The workflow finishes at END.

In a production system, these simple functions would typically use real LLMs, tools, APIs, persistence, permission checks, retries, and observability.

LangGraph also supports subgraphs, which are useful when a specialized workflow needs to operate as part of a larger agent architecture.

Summary

Multi-agent systems use multiple agents to solve problems through specialization, communication, coordination, and collaboration.

Traditional multi-agent systems involve autonomous entities that may cooperate or compete, while modern LLM-based multi-agent architectures typically use specialized AI agents working toward a shared objective.

Common architectures include specialized agents, hierarchical systems, supervisors, routers, planner-executor patterns, and custom workflows.

The key principle is simple: more agents do not automatically create a better system. A single agent with the right tools is often simpler, faster, and cheaper.

Use multiple agents when specialization, context isolation, parallel execution, or clear responsibility boundaries provide a real advantage.