Types of LLM Integration

Types of LLM Integration refer to different ways of connecting applications with AI models to build intelligent and scalable systems

These include approaches like Direct API, Multi-Provider, Local LLM, Framework-Based, RAG, Hybrid, and Agent-Based integration

1. Direct API Integration

Simple and quick to implement with minimal setup
Directly connects your application with a single AI provider using APIs
Best suited for small projects or initial development stages

Example (Spring Boot using RestTemplate)

import org.springframework.http.*;
import org.springframework.web.client.RestTemplate;

public class OpenAIService {

    private static final String API_URL = "https://api.openai.com/v1/chat/completions";
    private static final String API_KEY = "YOUR_API_KEY";

    public String getResponse(String userInput) {

        RestTemplate restTemplate = new RestTemplate();

        String requestBody = """
        {
          "model": "gpt-4o-mini",
          "messages": [
            {
              "role": "user",
              "content": "%s"
            }
          ]
        }
        """.formatted(userInput);

        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_JSON);
        headers.setBearerAuth(API_KEY);

        HttpEntity<String> entity = new HttpEntity<>(requestBody, headers);

        ResponseEntity<String> response = restTemplate.exchange(
                API_URL,
                HttpMethod.POST,
                entity,
                String.class
        );

        return response.getBody();
    }
}

2. Multi-Provider Integration

Allows integration with multiple AI providers at the same time
Enables switching between providers based on performance, cost, or availability
Helps avoid dependency on a single provider and improves reliability

Example

public class LLMRouterService {

    public String getResponse(String provider, String input) {

        if (provider.equals("openai")) {
            return callOpenAI(input);

        } else if (provider.equals("gemini")) {
            return callGemini(input);

        } else {
            return "Unsupported provider";
        }
    }

    private String callOpenAI(String input) {
        return "Response from OpenAI for: " + input;
    }

    private String callGemini(String input) {
        return "Response from Gemini for: " + input;
    }
}

3. Local LLM Integration

Runs AI models locally without depending on external APIs
Suitable for offline usage, better data privacy, and full control over the model
Common examples include models like LLaMA and Mistral

Example (Ollama API)

import org.springframework.web.client.RestTemplate;

public class LocalLLMService {

    public String askLocalModel(String prompt) {

        RestTemplate restTemplate = new RestTemplate();

        String url = "<http://localhost:11434/api/generate>";

        String request = """
        {
          "model": "llama3",
          "prompt": "%s",
          "stream": false
        }
        """.formatted(prompt);

        return restTemplate.postForObject(url, request, String.class);
    }
}

4. Framework-Based Integration

Uses frameworks like Spring AI or LangChain to simplify integration
Provides built-in features for prompt handling, chaining, and orchestration
Makes development faster, cleaner, and more scalable

Example

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.stereotype.Service;

@Service
public class AIService {

    private final ChatClient chatClient;

    public AIService(ChatClient chatClient) {
        this.chatClient = chatClient;
    }

    public String getAnswer(String prompt) {

        return chatClient.prompt()
                .user(prompt)
                .call()
                .content();
    }
}

Example Controller

import org.springframework.web.bind.annotation.*;

@RestController
@RequestMapping("/ai")
public class AIController {

    private final AIService aiService;

    public AIController(AIService aiService) {
        this.aiService = aiService;
    }

    @GetMapping("/ask")
    public String ask(@RequestParam String q) {
        return aiService.getAnswer(q);
    }
}

5. Hybrid LLM Integration

Combines cloud LLMs and local LLMs in a single system
Routes requests based on data sensitivity, cost, or performance needs
Balances privacy, speed, and scalability

Example (Routing Logic)

public class HybridLLMService {

    private final LocalLLMService localLLMService;
    private final OpenAIService openAIService;

    public HybridLLMService(LocalLLMService localLLMService, OpenAIService openAIService) {
        this.localLLMService = localLLMService;
        this.openAIService = openAIService;
    }

    public String getResponse(String input, boolean sensitiveData) {

        if (sensitiveData) {
            return localLLMService.askLocalModel(input);
        } else {
            return openAIService.getResponse(input);
        }
    }
}

6. RAG-Based Integration (Retrieval Augmented Generation)

Combines LLM with external data sources like databases, PDFs, or documents
Allows AI to answer based on your private or custom data
Very important for enterprise-grade AI systems

Example (Simple Retrieval + LLM Flow)

public class RAGService {

    private final DocumentService documentService;
    private final OpenAIService openAIService;

    public RAGService(DocumentService documentService, OpenAIService openAIService) {
        this.documentService = documentService;
        this.openAIService = openAIService;
    }

    public String askQuestion(String question) {

        String context = documentService.searchRelevantData(question);

        String finalPrompt = "Answer based on this context: " + context +
                             " Question: " + question;

        return openAIService.getResponse(finalPrompt);
    }
}

Example Use Case Flow

User asks question
System fetches relevant document data from database or files
LLM generates answer using that context

7. Agent-Based Integration

AI acts as an autonomous agent that can take actions
Can call APIs, execute workflows, and make decisions
Goes beyond simple text generation

Example (Tool Calling Agent Style)

public class AIAgentService {

    private final EmailService emailService;
    private final MeetingService meetingService;

    public String processTask(String task) {

        if (task.contains("email")) {
            emailService.sendEmail("test@example.com", "Hello from AI");
            return "Email sent successfully";
        }

        if (task.contains("meeting")) {
            meetingService.scheduleMeeting("Team Sync");
            return "Meeting scheduled successfully";
        }

        return "No valid action found";
    }
}

Difference between Direct API vs Framework-Based Integration

Feature	Direct API Integration	Framework-Based Integration
Approach	Directly calls LLM APIs	Uses frameworks like Spring AI or LangChain
Complexity	Low to Medium	Low (but structured)
Code Handling	Manual request/response handling	Abstracted and simplified
Development Speed	Slower	Faster
Scalability	Limited	High
Best For	Small or simple projects	Production-grade applications

Difference between RAG vs Normal LLM Integration

Feature	Normal LLM Integration	RAG (Retrieval Augmented Generation)
Data Source	Only trained model data	External data + LLM
Knowledge Base	Static knowledge	Dynamic + custom knowledge
Accuracy for business data	Low	High
Use of Documents/DB	Not used	Uses PDFs, DB, files, APIs
Complexity	Simple	Medium to High
Best For	General chatbot	Enterprise AI systems

Production Best Practices

Always store API keys in environment variables
Add retry and fallback mechanisms
Use caching to reduce API cost
Implement rate limiting for safety
Monitor token usage for cost control
Log requests for debugging and analytics

Conclusion

LLM integration is a modern approach to build intelligent applications by connecting your system with powerful AI providers.

With proper architecture like Direct API, Multi-Provider, Local LLM, or Framework-Based integration, developers can build scalable, secure, and production-ready AI systems.