Author
In high-throughput microservices, threads are a limited resource. Improper management can lead to thread exhaustion, increased latency, or system crashes. Backpressure prevents the system from being overwhelmed, ensuring stability and predictable performance.
Thread exhaustion happens when all threads in a pool are busy, and new tasks cannot be processed.
Common in I/O-heavy microservices, especially with synchronous calls.
Symptoms:
OutOfMemoryError if tasks pile upCauses:
Traditional Threads: Each consumes ~1MB of memory, limiting scalability.
Virtual Threads (Java 21+): Extremely lightweight (~1KB per thread), perfect for high-concurrency, I/O-bound workloads. However, always monitor external resources like database or API connections to avoid bottlenecks.
Backpressure helps prevent thread exhaustion by slowing down or rejecting requests when a service is overloaded. This ensures system stability and predictable performance.
Common Strategies:
Proper thread pool configuration is essential for managing asynchronous and concurrent tasks in microservices.
a) Bounded Thread Pools
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(20);
executor.setMaxPoolSize(50);
executor.setQueueCapacity(100); // Prevent unbounded queue growth
executor.setThreadNamePrefix("AppThread-");
executor.initialize();
b) RejectedExecutionHandler
Handles tasks that cannot be executed because the pool and queue are full:
executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
Common Policies:
Blocking operations can exhaust threads and increase latency. To prevent this:
WebClient (Spring WebFlux) instead of RestTemplateR2DBC for reactive database access instead of JDBCExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();
CompletableFuture.supplyAsync(() -> serviceCall(), executor);
Proper queue handling prevents memory pressure and ensures predictable latency:
This approach maintains stable throughput and prevents thread exhaustion under high load.
To protect microservices during traffic spikes, control incoming requests using:
Keep track of system health and thread usage to avoid thread exhaustion:
Key Metrics:
Monitoring Tools:
// 1. Configure a bounded thread pool for async tasks
@Configuration
@EnableAsync
public class AsyncConfig {
@Bean(name = "paymentExecutor")
public Executor paymentExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(20);
executor.setMaxPoolSize(50);
executor.setQueueCapacity(50); // Bounded queue
executor.setThreadNamePrefix("Payment-");
executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
executor.initialize();
return executor;
}
}
// 2. PaymentService with async processing
@Service
public class PaymentService {
@Async("paymentExecutor")
public CompletableFuture<String> processPayment(String orderId) {
try {
// Simulate I/O-bound payment processing
Thread.sleep(2000); // e.g., call to external payment API
return CompletableFuture.completedFuture("Payment processed for order " + orderId);
} catch (InterruptedException e) {
return CompletableFuture.completedFuture("Payment failed for order " + orderId);
}
}
}
// 3. Controller with backpressure handling
@RestController
@RequestMapping("/payments")
public class PaymentController {
private final PaymentService paymentService;
private final Semaphore semaphore = new Semaphore(50); // Backpressure control
public PaymentController(PaymentService paymentService) {
this.paymentService = paymentService;
}
@PostMapping("/{orderId}")
public ResponseEntity<String> pay(@PathVariable String orderId) {
// Apply backpressure: reject requests if max concurrent tasks reached
if (!semaphore.tryAcquire()) {
return ResponseEntity.status(429).body("Too many requests, try later");
}
CompletableFuture<String> result = paymentService.processPayment(orderId)
.whenComplete((res, ex) -> semaphore.release());
return ResponseEntity.ok("Payment request accepted for order " + orderId);
}
}
Scenario: High-concurrency payment service under heavy load
Implementation:
Benefits:
Thread exhaustion can severely impact microservice performance. Proper thread pool sizing and backpressure mechanisms prevent overload. Using non-blocking I/O or Virtual Threads allows handling thousands of concurrent requests efficiently. Monitoring and alerting ensures visibility into queues, active threads, and resource usage. Combining these strategies makes microservices resilient, scalable, and responsive, even under high traffic.