Batch Processing

Batch processing is the technique of handling a large amount of data in chunks (batches) instead of processing records one by one. In Spring Boot and Hibernate, it is widely used to optimize bulk insert, update, and delete operations by reducing database calls and controlling memory usage.

In simple words: Batch processing groups multiple database operations and executes them efficiently together.

---

Why Batch Processing Is Important

• Improves performance for bulk operations
• Reduces database network calls
• Controls memory usage
• Speeds up inserts, updates, and deletes
• Essential for data migration and large imports

---

Common Use Cases

• Bulk data import (CSV, Excel, APIs)
• Data migration scripts
• Scheduled jobs (cron / batch jobs)
• Processing logs or transactions
• ETL (Extract, Transform, Load) jobs

---

What is Batch Processing in Hibernate?

Hibernate supports JDBC batching, which groups multiple SQL statements and sends them together instead of executing one statement per record.

Enable Batch Processing

application.properties

spring.jpa.properties.hibernate.jdbc.batch_size=50
spring.jpa.properties.hibernate.order_inserts=true
spring.jpa.properties.hibernate.order_updates=true
spring.jpa.properties.hibernate.generate_statistics=true

• Groups SQL statements for better performance
• Optimizes batch execution efficiency

---

Batch Operations

Batch operations handle large datasets efficiently by grouping multiple operations into batches and periodically flushing the persistence context to the database.

Key Concepts: flush() and clear()

• flush() → Executes pending SQL statements
• clear() → Clears the first-level (L1) cache to prevent memory growth
• Together, they prevent OutOfMemoryError during large batch operations.

---

Batch Insert Example

@Transactional
publicvoidsaveUsers(List<User> users) {
for (inti=0; i < users.size(); i++) {
        entityManager.persist(users.get(i));
if (i %50 ==0) {
            entityManager.flush();
            entityManager.clear();
        }
    }
}

• persist() queues SQL statements
• flush() sends the batch to the DB
• clear() prevents memory issues

---

Batch Update Example

@Transactional
publicvoidupdatePrices(List<Product> products) {
for (inti=0; i < products.size(); i++) {
        products.get(i).setPrice(products.get(i).getPrice() +10);
if (i %50 ==0) {
            entityManager.flush();
            entityManager.clear();
        }
    }
}

• Updates are applied in memory first
• flush() commits updates in batches
• clear() keeps memory under control

---

Batch Delete Example

@Transactional
publicvoiddeleteOldOrders(List<Order> orders) {
for (inti=0; i < orders.size(); i++) {
        entityManager.remove(orders.get(i));
if (i %50 ==0) {
            entityManager.flush();
            entityManager.clear();
        }
    }
}

• remove() marks entities for deletion
• Batch delete is faster than deleting one by one
• Memory is kept under control with flush() and clear()

---

Bulk Update / Delete Using Query

Queryquery= entityManager.createQuery(
"UPDATE User u SET u.active = false"
);
query.executeUpdate();

• Faster than entity-level updates
• Skips dirty checking and persistence context

---

Batch Processing with Spring Data JPA

@Transactional
publicvoidsaveUsers(List<User> users) {
    userRepository.saveAll(users);
}

⚠️ saveAll() only uses batching if hibernate.jdbc.batch_size is configured.

---

Batch Size Selection

Size	Effect
Small (<30)	Too many DB calls
Medium (30–100)	Balanced performance & memory
Large (>1000)	High memory usage, risk of OOM

Recommended: 30–100 per batch

---

Stateless vs Stateful Sessions

Hibernate provides two types of sessions:

1. Stateful Session (Default Session)

• Maintains first-level cache (L1 cache) per session.
• Tracks entity states (dirty checking) automatically.
• Supports cascading operations.
• Ideal for normal CRUD operations where entity state tracking is needed.

Example:

Sessionsession= sessionFactory.openSession();
Transactiontx= session.beginTransaction();

Useruser= session.get(User.class,1L);// tracked in L1 cache
user.setName("John Doe");

tx.commit();// flushes changes
session.close();

2. Stateless Session

• Does not maintain L1 cache.
• No dirty checking or cascading.
• Extremely fast for bulk inserts, updates, or deletes.
• Recommended for large batch processing.

Example:

StatelessSessionsession= sessionFactory.openStatelessSession();
Transactiontx= session.beginTransaction();

for (User user : users) {
    session.insert(user);// direct insert
}

tx.commit();
session.close();

---

Batch Processing vs Normal Processing

Aspect	Normal Processing	Batch Processing
DB Calls	Executes one SQL statement per record	Groups multiple operations into batches, reducing database round trips
Performance	Slower for large datasets due to many DB hits	Faster because fewer DB calls and optimized execution
Memory Usage	High, as the persistence context grows with each record	Controlled by using flush() and clear() periodically
Best For	Small datasets or real-time operations	Large datasets, bulk inserts/updates/deletes, data migration

---

Common Mistakes

• Forgetting flush() and clear()
• Using extremely large batch sizes
• Running batch jobs inside web requests
• Keeping L2 cache enabled during batch jobs
• Logging SQL statements for every operation

---

Best Practices

• Tune batch size based on your DB and memory
• Flush & clear regularly
• Disable SQL logging during batch jobs
• Avoid cascading large object graphs
• Use pagination for reading large datasets
• Run batch jobs in dedicated services

---

When NOT to Use Batch Processing

• Small datasets
• Real-time user requests
• Highly complex entity relationships
• Operations requiring immediate consistency

---

Conclusion

• Batch processing is essential for high-performance data handling
• Hibernate supports batching via JDBC batching and stateless sessions
• Proper configuration, batch size selection, and session management are crucial
• Best suited for bulk inserts, updates, deletes, and data migrations