Agent Technologies and Frameworks

Introduction

Building an AI agent requires more than a Large Language Model (LLM). An LLM can understand language and generate responses, but real-world agents also need technologies to access data, use tools, maintain context, and complete multi-step tasks.

For example, an AI agent may use:

• APIs and tools to interact with external systems.

• Vector databases to retrieve relevant information from documents.

• Memory systems to maintain useful context.

• Agent frameworks to coordinate workflows, tools, and agent behavior.

Together, these technologies help developers build AI assistants, research agents, coding assistants, customer-support systems, and business automation applications.

In this chapter, we will explore the main technologies behind Agentic AI and the role of popular AI agent frameworks in building modern agent-based applications.

Technologies Behind Agentic AI

An AI agent combines multiple technologies to understand requests, access information, make decisions, use tools, and complete tasks.

The LLM is a core component, but depending on the use case, an agent may also need:

• Large Language Models (LLMs) — Understand instructions and generate responses.

• APIs and Tool Integration — Allow agents to interact with external applications and services.

• Vector Databases — Store and retrieve embeddings for semantic search and RAG-based applications.

• Knowledge Graphs — Represent entities and relationships as structured knowledge.

• Memory Systems — Maintain relevant information across interactions or tasks.

• Planning and Reasoning — Help agents break complex goals into steps and decide what action to take next.

• AI Agent Frameworks — Provide building blocks for tools, workflows, state, orchestration, and agent execution.

These technologies can be combined based on the application's requirements. Not every AI agent needs every technology; a good agent architecture uses only the components required for its tasks.

APIs and Tool Integration

A Large Language Model (LLM) can understand instructions and generate responses, but it cannot perform every real-world action by itself. An AI agent uses APIs and tools to interact with external systems, retrieve information, and take actions.

An API (Application Programming Interface) provides a way for the agent to communicate with another application or service.

For example, an AI agent may use APIs to:

• Check current weather

• Search the web

• Send emails

• Schedule meetings

• Process payments

• Access business applications

• Retrieve customer information

Agents can also work with tools such as databases, file systems, search services, calculators, code execution environments, and cloud storage.

Example

Suppose you ask:

"Email today's sales report to my manager."

The agent could perform the following steps:

1. Retrieve the sales data from a database.

2. Generate a short summary of the report.

3. Use an email tool or API to send it.

4. Return the result of the operation to the user.

Here, the LLM helps understand the request and decide which actions are needed, while the tools actually perform those actions.

This ability to select and use external tools is commonly called tool use or tool calling, and it is a key capability of modern AI agents.

Vector Databases

An AI agent may need information that is not available in the LLM's current context. One way to retrieve relevant information from a large knowledge base is through vector search.

Content such as documents can be converted into embeddings—numerical representations that capture useful semantic information. These embeddings can then be stored and searched using a vector database or another system that supports vector search.

A simplified flow looks like this:

Documents
   ↓
Create Embeddings
   ↓
Store in Vector Database

User Question
   ↓
Create Query Embedding
   ↓
Similarity Search
   ↓
Retrieve Relevant Content
   ↓
Provide Context to LLM
   ↓
Generate Response

Example

Suppose a company has thousands of internal documents.

When an employee asks:

"What is our refund policy for annual subscriptions?"

The system can convert the question into an embedding and search for semantically similar document chunks. This means relevant content can be found even when the document does not use exactly the same words as the user's question.

Popular technologies used for vector search include Pinecone, Weaviate, Milvus, Qdrant, and Chroma.

Vector search is commonly used in Retrieval-Augmented Generation (RAG) systems. The retrieved information is added to the LLM's context before generation, helping the agent answer using relevant external knowledge rather than relying only on the model's built-in knowledge.

Knowledge Graphs

A Knowledge Graph organizes information as entities and relationships. Instead of treating information as isolated records, it connects related entities so an AI agent can understand how they are related.

For example:

Aman ──works_at──► ABC Company
                      │
                   develops
                      ▼
                  Product X
                      │
                  belongs_to
                      ▼
                   Category

Here, Aman, ABC Company, Product X, and Category are entities, while works_at, develops, and belongs_to represent relationships between them.

Knowledge Graphs can help AI agents:

• Understand relationships between entities

• Retrieve connected information

• Combine information from multiple sources

• Answer questions that depend on relationships and context

They are commonly useful in areas such as:

• Enterprise search

• Recommendation systems

• Healthcare

• Financial analysis

• Customer support

• Business intelligence

When combined with Large Language Models (LLMs), a knowledge graph can provide structured and connected information that the model can use while answering questions. This can be especially useful when an agent needs to reason over entities, relationships, and domain-specific knowledge rather than relying only on unstructured text.

Popular AI Agent Frameworks

Building an AI agent from scratch requires developers to manage several components, such as LLMs, prompts, tools, state, workflows, and external integrations.

AI agent frameworks simplify this process by providing reusable components and abstractions for building and coordinating agent-based applications. Instead of implementing every capability from scratch, developers can focus more on the application's actual logic.

Depending on the framework, common capabilities may include:

• Tool integration and tool calling

• Agent state and memory management

• Workflow and task orchestration

• Multi-agent coordination

• LLM and model integration

• Observability, testing, and debugging support

Different frameworks focus on different problems. Some are useful for building simple tool-using agents, while others are designed for multi-agent systems, structured workflows, or production AI applications.

In the following sections, we will explore some popular AI agent frameworks, understand what each one provides, and see when it can be useful.

LangChain

LangChain is an open-source framework for building applications powered by Large Language Models (LLMs).

It provides reusable components and integrations for working with models, prompts, tools, retrieval systems, and external data sources. This makes it easier to connect an LLM with the other components required by an AI application.

For example, a LangChain application can:

• Receive a user request

• Retrieve relevant information from external data

• Let the model call available tools

• Process the results

• Generate the final response

LangChain is commonly used for building RAG applications, AI assistants, document-based applications, and tool-using agents.

Common Use Cases

• AI Assistants

• RAG Applications

• Document Question Answering

• Tool-Using Agents

• LLM-Powered Applications

LangGraph

LangGraph is a framework for building stateful, long-running agents and workflows using a graph-based execution model.

Instead of limiting an agent to a simple sequence of steps, LangGraph allows a workflow to contain different paths, decisions, loops, and state.

A simple workflow might look like:

User Request
     ↓
   Agent
     ↓
Need a Tool?
   /     \
 Yes      No
  ↓        ↓
Use Tool  Respond
  ↓
Agent
  ↓
Respond

The ability to return to an earlier step is useful for agents that may need to use multiple tools, evaluate results, and continue working until a task is complete.

LangGraph is particularly useful when an application needs more control over agent execution, state, and multi-step workflows.

Common Use Cases

• Multi-Step AI Agents

• Stateful Agent Workflows

• Multi-Agent Systems

• Long-Running Workflows

• Human-in-the-Loop Applications

• Decision-Based Workflows

CrewAI

CrewAI is an open-source framework for building multi-agent systems where multiple AI agents collaborate to complete a task.

Each agent can be given a specific role, goal, and set of tools. The agents can then work together as a team, with different agents handling different parts of a workflow.

For example:

User Task
   ↓
Research Agent
   ↓
Writer Agent
   ↓
Reviewer Agent
   ↓
Final Output

Here, the Research Agent collects information, the Writer Agent creates the content, and the Reviewer Agent checks the result.

CrewAI is useful when a complex task can be divided into specialized responsibilities handled by multiple collaborating agents.

Common Use Cases

• Research Automation

• Content Creation

• Business Workflows

• Multi-Agent Collaboration

• Task Automation

AutoGen

AutoGen is an open-source framework from Microsoft for building agent-based and multi-agent applications.

It allows developers to create agents that can communicate with each other, use tools, and collaborate on tasks. Different agents can be assigned different responsibilities depending on the workflow.

For example, a software-development workflow might involve:

User Request
     ↓
Planning Agent
     ↓
Coding Agent
     ↓
Review Agent
     ↓
Final Result

AutoGen is useful for applications where multiple agents need to exchange information, coordinate actions, and work together on multi-step tasks.

Common Use Cases

• Multi-Agent Applications

• Research Assistants

• Coding Workflows

• Task Automation

• Collaborative AI Systems

OpenAI Agents SDK

The OpenAI Agents SDK is an open-source framework for building agentic AI applications. It provides a small set of building blocks for creating agents that can use tools, maintain context, and collaborate with other agents.

An agent can be configured with instructions and tools, while the SDK manages the agent loop and tool execution. It also supports capabilities such as handoffs, guardrails, sessions, human-in-the-loop workflows, and tracing. Sessions can provide persistent conversation context across agent runs.

For example:

User Request
     ↓
OpenAI Agent
     ↓
Choose Tool
     ↓
Execute Tool
     ↓
Process Result
     ↓
Final Response

The SDK is useful when developers want a lightweight way to build tool-using or multi-agent applications while having the runtime manage common agent behavior.

Common Use Cases

• AI Assistants

• Customer Support Agents

• Tool-Using Agents

• Multi-Agent Workflows

• Task Automation

• Voice Agents

OpenAI Agents SDK documentation

Semantic Kernel

Semantic Kernel is an open-source SDK from Microsoft for integrating AI models with application code, services, and external tools.

A central concept in Semantic Kernel is the Kernel, which manages AI services and plugins used by the application. Plugins expose functions that allow AI applications and agents to interact with existing APIs, databases, and business logic.

Semantic Kernel also provides an Agent Framework for creating agents and supporting agent-based workflows, including scenarios where multiple agents collaborate.

A simplified architecture looks like:

Application
    ↓
Semantic Kernel
   /       \
  ↓         ↓
AI Model   Plugins
             ↓
      APIs / Services

Semantic Kernel is especially useful when developers need to integrate AI capabilities with existing application code and enterprise systems.

Common Use Cases

• Enterprise AI Applications

• AI Assistants

• Business Process Automation

• Tool and API Integration

• Multi-Agent Applications

Semantic Kernel documentation

LlamaIndex

LlamaIndex is a framework for building LLM-powered applications and AI agents that work with external data.

It provides tools for connecting data from sources such as documents, databases, APIs, and cloud services, then processing and retrieving the relevant information when an application needs it.

A simple flow looks like this:

External Data
     ↓
LlamaIndex
     ↓
Index / Retrieve
     ↓
Relevant Context
     ↓
LLM / AI Agent
     ↓
Response

LlamaIndex is especially useful for Retrieval-Augmented Generation (RAG), where an application retrieves relevant information from its own data before asking the LLM to generate a response.

It can also be used in agentic applications where agents need to search, retrieve, and work with private or domain-specific knowledge.

Common Use Cases

• RAG Applications

• Enterprise Search

• Knowledge Management

• Document Intelligence

• Data-Driven AI Agents

• Research Assistants

Choosing the Right Framework

There is no single AI agent framework that is best for every project. The right choice depends on factors such as application requirements, workflow complexity, data sources, tool integration, and whether you need single-agent or multi-agent workflows.

Here is a simple comparison:

For beginners, LangChain can be a useful starting point for learning common LLM application concepts and integrations. However, the best framework should be chosen based on the problem rather than popularity.

For example, LangGraph is useful when you need explicit control over state and multi-step execution, CrewAI focuses on role-based multi-agent collaboration, and LlamaIndex is a strong choice when the application mainly needs to work with external or private data.

In real-world projects, these technologies are also not always mutually exclusive. For example, an application may use an agent framework for orchestration while using a separate retrieval system for RAG.

Summary

AI agents combine multiple technologies to understand goals, access information, make decisions, use tools, and complete tasks.

Technologies such as APIs and tool integration allow agents to interact with external systems, while vector databases and knowledge graphs help them work with external knowledge and connected information. Depending on the application, agents may also use memory, planning, reasoning, and workflow orchestration.

Frameworks such as LangChain, LangGraph, CrewAI, AutoGen, OpenAI Agents SDK, Semantic Kernel, and LlamaIndex provide reusable components that simplify the development of agentic applications.

There is no single framework that is best for every project. The right choice depends on the application's data, tools, workflow complexity, state-management needs, and single-agent or multi-agent architecture.

Understanding these technologies and frameworks provides a strong foundation for designing reliable, scalable, and production-ready AI agent systems.