Docker Storage Layers & Union File System

Docker uses a smart layered storage system combined with Union File System (OverlayFS) to make images lightweight, reusable, and fast.

This is one of the most important Docker concepts because it explains how images are built, stored, and optimized.

What are Docker Image Layers?

Docker images are built using multiple read-only layers that are stacked on top of each other like a tree structure.

Each line (instruction) in a Dockerfile creates a new layer in the image.

Each layer represents a specific change (like installing software, copying files, or setting configuration)
Layers are stacked in a fixed order (bottom to top)
Layers are reusable across different images
Docker uses caching to avoid rebuilding unchanged layers
Layers help make images lightweight and faster to build

? In simple words: A Docker image is a combination of multiple stacked layers.

Example of Docker Image Layers

FROM openjdk:21
WORKDIR /app
COPY target/app.jar app.jar
ENTRYPOINT ["java","-jar","app.jar"]

How layers are created:

FROM openjdk:21 → Base layer (OS + Java environment)
WORKDIR /app → Creates a working directory layer
COPY target/app.jar app.jar → Adds application code layer
ENTRYPOINT ["java","-jar","app.jar"] → Final runtime configuration layer

Layer Structure in Docker

Docker images are built in a layered format. Each layer adds a new change on top of the previous one.

Base Image Layer (OS)
        ↓
Runtime Layer (Java / Node / Python)
        ↓
Application Layer (Code)
        ↓
Configuration Layer
        ↓
Writable Container Layer

Read-Only vs Writable Layers

Docker layers are mainly divided into two types:

Feature	Read-Only Layers (Image Layers)	Writable Layer (Container Layer)
Purpose	Forms the base Docker image	Stores runtime changes during container execution
Creation Time	Created during image build process	Created when container starts running
Modifiability	Cannot be modified once created	Can be modified during container runtime
Data Type	Static data	Dynamic data
Sharing	Shared across multiple containers	Unique for each container
Storage	Stored in Docker cache system	Stored inside container filesystem
Lifecycle	Remains until image is deleted	Removed when container is deleted

Layer Caching Mechanism

Docker uses layer caching to improve build speed and efficiency.

How it works:

If a layer has not changed → Docker reuses it from cache
If a layer changes → Docker rebuilds that layer and all layers after it
Previous unchanged layers are reused automatically

Example:

If you only change application code:

Base image is NOT downloaded again
Only the COPY and next layers are rebuilt
Build becomes much faster

? This is why Docker is very efficient in CI/CD pipelines.

Copy-on-Write (CoW) Concept

Docker uses Copy-on-Write (CoW) to manage data efficiently.

How it works:

Base image layers are shared and remain read-only
When a file is modified inside a container:
- Docker does NOT change the original layer
- Instead, it copies the file into the writable layer
The container uses the modified version from the writable layer

? In simple words: Docker never edits original data — it creates a copy when changes are needed.

Union File System (OverlayFS)

Docker uses Union File System (OverlayFS) to combine multiple image layers into a single unified filesystem view.

This is what makes Docker feel like a complete operating system even though it is made of many layers.

What Union File System Does

Combines all Docker image layers into one system view
Makes the container think it has a single filesystem
Keeps all layers separate internally for efficiency
Improves performance and reduces duplication

? In simple words: Multiple layers are merged and shown as one complete system inside the container.

How Union File System Works

Docker uses a layered stacking approach:

Multiple layers are stacked on top of each other
Docker merges them logically using OverlayFS
Lower layers remain read-only
The top layer is writable (container layer)
Any changes are written only to the top layer

Docker Layer Flow

Docker follows a structured flow where image layers are combined and used to run containers.

Docker Image Layers (Read-Only)
        ↓
Union File System (OverlayFS)
        ↓
Container Writable Layer
        ↓
Running Application

Inspecting Image Layers (docker history)

Docker provides a command to view how an image is built step-by-step using layers.

This is very useful for debugging and understanding image structure.

Command:

docker history <image_name>

Example:

docker history nginx

Why docker history is Useful

Helps understand image structure
Useful for optimization and debugging
Shows how Docker caching works
Helps reduce image size in production

How Layers Improve Performance

Docker layers improve performance by reusing cached layers, which makes builds and deployments faster and avoids unnecessary work.

Faster Builds: Only changed layers are rebuilt, others are reused.
Faster Deployments: Existing layers are reused, reducing download time.
Efficient Storage: Common layers are shared, saving disk space.
Better Caching: Previous builds are reused to speed up CI/CD pipelines.

Best Practices for Layer Optimization

Docker layer optimization helps reduce image size, improve build speed, and make deployments more efficient.

Minimize Layers: Combine multiple Dockerfile commands into one where possible.
Use Multi-Stage Builds: Keep final images small and production-ready.
Order Instructions Properly: Place stable layers (like base image and dependencies) first for better caching.
Use .dockerignore: Exclude unnecessary files from the build context.
Avoid Frequent Base Layer Changes: So Docker cache can be reused effectively.

Conclusion

Docker storage layers and Union File System (OverlayFS) make Docker highly efficient by enabling layer reuse and reducing duplication.

Fast
Lightweight
Efficient
Reusable

? This layered architecture is the key reason Docker is faster and more resource-efficient than traditional virtual machines.