
Durgesh Tiwari
Author
Suppose you want to share a long piece of text, code, or logs with someone. Instead of sending everything in a chat, you can store it in a Pastebin-like service and share a short URL.
For example:
<https://paste.example/X7aP9k>At first, the system looks simple:
Text → Store → Generate Link → ShareBut as the service grows, we need to think about storage, unique paste IDs, caching, expiration, and security.
In this Pastebin System Design, we will start with a simple solution and improve it step by step as new problems appear.

We are designing a Pastebin-like text sharing service where users can store text or code and get a short URL to share it.
The basic flow is:
Create Paste → Store Content → Generate URL → ShareWe will focus on text and code sharing, not large file storage.
Our system should support these core features:
Create a paste: Users can submit text or source code and receive a unique shareable URL.
Read a paste: Anyone with the required access can open the URL and view the stored content.
Anonymous pastes: Users can create basic pastes without creating an account.
Manage pastes: Logged-in users can manage or delete the pastes they own.
We will add features such as visibility and expiration as we build the design.
A paste can have one of three visibility levels:
Public: Anyone can view it, and it may appear in search or public pages.
Unlisted: It does not appear in public discovery. Users normally need the URL to find it.
Private: Only authorized users can access it.
An important difference is that unlisted does not mean private. Anyone who gets an unlisted URL may still be able to open and share it.
Private pastes require a real authorization check.

Users should be able to create temporary pastes that expire after a specific time, such as one hour, one day, or one week.
We can store an expiration time with the paste:
expires_at = 2026-09-09 10:30:00After this time, the system should stop serving the paste.
Users should also be able to delete the pastes they own. Once deleted, the paste should no longer be accessible.
Later, we will see how to handle expiration and deletion efficiently without scanning the entire database.
For developers, the service can provide a raw text view that returns only the original content:
GET /raw/X7aP9kFor source code, users can also select a programming language such as Python, Java, or JavaScript. We store the selected language as metadata and use it for syntax highlighting.
The original paste content remains unchanged.
To keep the design focused, we will not cover large file uploads, real-time collaborative editing, or version control.
Our goal is simple:
Create → Store → Share → Read → ExpireAlong with the features above, our Pastebin system should provide:
High availability: Users should still be able to access pastes when some servers fail.
Low latency: Reading a paste should be fast.
Durability: Stored pastes should survive server or disk failures according to their retention requirements.
Scalability: The system should handle millions of pastes and growing read traffic.
Security: Private pastes must only be available to authorized users.
The system is also naturally read-heavy. A paste is usually created once but can be viewed many times. This will become important when we design the caching and storage layers.
Before designing the architecture, let's estimate the expected traffic and storage.
Assume:
10 million new pastes per month
20 reads per paste on average
10 KB average paste size
With 10 million new pastes per month:
10,000,000 / 2,592,000
≈ 4 writes/secondThe average write traffic is small, although real traffic can have much higher peaks.
If each paste receives about 20 views:
10 million × 20
= 200 million reads/month
≈ 77 reads/secondThe bigger challenge is that traffic will not be evenly distributed. Most pastes may receive very little traffic, while one viral paste could receive thousands of requests per second.
This makes caching important for our design.
At an average size of 10 KB:
10 million × 10 KB
≈ 100 GB/month
≈ 1.2 TB/yearThe real storage requirement will be higher because of metadata, indexes, replicas, and backups.
These estimates tell us two important things:
The system is read-heavy.
Paste content will continue to grow over time.
Now let's start with the simplest storage design and improve it when needed.
For a small Pastebin service, we can store everything in a single database table:
PASTE
-------------------------
paste_id
user_id
content
title
language
visibility
created_at
expires_at
statusA paste might look like:
paste_id = X7aP9k
user_id = 451
content = "SELECT * FROM users;"
language = sql
visibility = unlisted
expires_at = nullReading a paste is then a simple lookup by paste_id.
This design is easy to build and works well at a small scale.
However, as the number and size of pastes grow, storing all content directly in the database becomes less attractive. Next, we can separate the small paste metadata from the larger paste content.
As the system grows, storing the entire paste in the database is not always the best choice.
A paste has two types of data:
Metadata: paste ID, owner, language, visibility, expiration time, and status.
Content: the actual text or code.
We can store them separately:
Metadata → Database
Content → Object StorageFor example, the database can store:
paste_id = X7aP9k
language = sql
visibility = unlisted
content_key = pastes/X7/aP9kThe content_key tells us where the actual text is stored in object storage.
Object storage works well here because paste content is usually written once and read many times. It also allows the metadata database and content storage to scale independently.
For a small system, storing everything in the database is still reasonable. We introduce object storage when the amount of content makes that separation useful.

Every paste needs a unique ID that becomes part of its URL:
<https://paste.example/X7aP9k>The ID should be short, unique, URL-friendly, and quick to generate.
A simple approach is to generate a random ID using:
a-z
A-Z
0-9This gives us 62 possible characters. With a 6-character ID:
62^6 ≈ 56 billion combinationsExample IDs might look like:
a9Km2P
7BxQ1z
mN8kL4Random IDs are not guaranteed to be unique. Two requests could generate the same ID.
To handle this safely, the database should enforce a unique constraint on paste_id.
The flow is simple:
Generate ID
↓
Try to Save
↓
Collision?
↙ ↘
No Yes
↓ ↓
Save Generate AgainIf collisions become more common as the service grows, we can increase the ID length.
This keeps ID generation simple while still preventing one paste from overwriting another.

Another option is to generate a globally unique numeric ID and encode it using Base62.
For example:
987654321 → Base62 → 4gfFC3This avoids random collisions because the underlying ID generator guarantees uniqueness across servers.
Both approaches are reasonable:
Random IDs: Simple, but require a uniqueness check and retry on collision.
Distributed IDs: Guarantee unique IDs, but require an additional ID-generation mechanism.
For our design, we will use random Base62 IDs with a database uniqueness constraint because they are simple and naturally harder to enumerate.
Random-looking IDs are especially useful for unlisted pastes, but they are not a security mechanism. Private pastes must still require authentication and authorization.
We only need a few core APIs:
API | Purpose |
|---|---|
| Create a new paste |
| Read a paste |
| Get only the raw content |
| Delete a paste |
POST /v1/pastesRequest:
{
"content": "console.log('Hello');",
"language": "javascript",
"visibility": "unlisted",
"expires_in": 86400
}Response:
{
"paste_id": "X7aP9k",
"url": "/p/X7aP9k",
"expires_at": "2026-09-09T10:00:00Z"
}For private pastes and deletion requests, the service must also verify that the current user has permission to access or modify the paste.
With the data model, ID strategy, and APIs defined, we can now connect these pieces into our first working architecture.
Now we can build our first scalable architecture.
┌──────────────┐
│ Client │
└──────┬───────┘
│
▼
┌──────────────┐
│ Load Balancer│
└──────┬───────┘
│
▼
┌──────────────┐
│Paste Service │
└──────┬───────┘
│
┌─────────┴─────────┐
▼ ▼
┌─────────────┐ ┌──────────────┐
│ Metadata DB │ │Object Storage│
└─────────────┘ └──────────────┘The Paste Service handles creation, retrieval, permission checks, expiration, and deletion.
When a user creates a paste:
Client
↓
Paste Service
↓
Validate request
↓
Generate paste ID
↓
Store content in Object Storage
↓
Store metadata in Database
↓
Return shareable URLBefore storing the paste, the service also validates fields such as the content size, visibility, and expiration settings.
One important issue is that the database and object storage are two separate systems.
For example, the content write may succeed while the metadata write fails:
Object Storage → Success
Metadata DB → FailureThis leaves an orphaned object that cannot be reached through the application.
We can handle this with retry-safe creation logic and a background cleanup process that removes orphaned objects.
The key point is that writes across two storage systems are not automatically atomic.

When a user opens a paste, the service first reads its metadata.
Client
↓
Paste Service
↓
Metadata DB
↓
Check visibility, status, and expiration
↓
Object Storage
↓
Return PasteThe metadata tells us whether the paste exists, whether it has expired or been deleted, whether the user can access it, and where its content is stored.
This design works, but every request may require both a database lookup and an object-storage read.
That becomes inefficient when the same paste is requested repeatedly.
Because Pastebin is read-heavy, we can place a distributed cache in front of the storage systems.
Client
↓
Paste Service
↓
Cache
↙ ↘
Hit Miss
↓ ↓
Return Metadata DB
+ Object Storage
↓
Update Cache
↓
Return PasteFor suitable public and unlisted pastes, we can cache the complete read response:
Key: paste:X7aP9k
Value:
- content
- title
- language
- visibility
- expires_atOn a cache hit, the service can return the paste without accessing the database or object storage.
This significantly reduces storage traffic for frequently viewed pastes.

Paste traffic can be highly uneven. Most pastes may receive very few views, while one viral paste can suddenly receive thousands of requests per second.
Because we already use the cache-aside pattern, popular pastes will usually be served directly from the distributed cache instead of repeatedly hitting the metadata database and object storage.
Requests
↓
Distributed Cache
↓
Return PasteFor extremely popular public pastes, we can later add a CDN to serve content closer to users and reduce load on our servers even further.
A problem appears when a popular cache entry expires while many requests arrive at the same time.
If every request sees a cache miss, they may all query the storage layer together. This is called a cache stampede.
We can reduce it using:
Request coalescing: Allow one request to refresh the cache while others wait.
Stale-while-revalidate: Temporarily serve a safe stale value while it is refreshed.
TTL jitter: Add small randomness to expiration times so many cache entries do not expire together.
Bots or users may repeatedly request paste IDs that do not exist.
Instead of querying the database every time, we can cache a not found result for a short period.
paste:not_found:aaaaaa → NOT_FOUNDThis protects the database from repeated invalid lookups.
The TTL should remain short so the cache does not incorrectly hide newly created data.

Suppose a paste has:
expires_at = 10:30If a request arrives at 10:31, the service must not return the paste even if its content still exists in object storage.
The read path therefore checks:
current_time >= expires_at ?If the paste has expired, access is denied immediately.
This gives us logical expiration. The actual content can be physically removed from storage later by a background cleanup process.

We do not need to physically delete every paste at the exact second it expires.
Instead, we separate:
Logical Expiration → Stop serving the paste
Physical Cleanup → Delete stored data laterThe read path already handles logical expiration using expires_at. A background worker can handle physical deletion asynchronously.
Expired Pastes
↓
Cleanup Workers
↙ ↘
Metadata Object StorageThis keeps cleanup work away from user requests and avoids large deletion spikes when many pastes expire at the same time.
The cleanup worker should not scan the entire database looking for expired records.
A simple approach is to maintain an index on expires_at and fetch expired pastes in batches:
expires_at <= current_timeAt larger scale, we could use delayed queues or time buckets, where pastes are grouped by their expiration window:
10:30 → Paste A, Paste B
10:31 → Paste C, Paste D
10:32 → Paste EWorkers only process the relevant bucket instead of searching through unrelated pastes.
The exact approach depends on the scale and how many temporary pastes the service stores.
User-requested deletion follows a similar pattern.
When an owner deletes a paste:
Authenticate User
↓
Verify Ownership
↓
Mark Paste Deleted
↓
Invalidate Cache
↓
Queue Storage + Search CleanupWe mark the paste as deleted first so new requests immediately stop returning it. Slower operations, such as removing the content from object storage and the search index, can happen asynchronously.
This keeps the user-facing delete operation fast while still cleaning up all copies of the paste.
Our deletion flow can use a soft delete first:
status = DELETEDThis immediately prevents new reads while keeping the metadata temporarily available for retries and background cleanup.
A worker can permanently remove the data later according to the product's retention policy.
For stricter privacy requirements, the system may require faster or immediate hard deletion instead.
Not every task needs to happen while the user waits for paste creation to finish.
For example, public search indexing and analytics can happen asynchronously:
Create Paste
↓
Save Required Data
↓
Publish Event
↓
Event Queue
↙ ↘
Search Analytics
Worker WorkerThis keeps the main request path fast and reduces dependencies on secondary systems.
If the product supports searching public pastes, a separate search index can handle full-text queries instead of the metadata database.
Only public and searchable pastes should be added to this index. Private and unlisted pastes must not accidentally become discoverable.
Because indexing happens asynchronously, a newly created paste may take a short time to appear in search. This eventual consistency is usually acceptable.
We should also avoid updating the main metadata row every time someone views a paste. A viral paste could otherwise create a large number of writes to the same record.
Instead:
Paste View → Event Queue → Analytics Worker → Analytics StoreThe displayed view count may be slightly delayed, which is generally acceptable for analytics.
Private pastes require an authorization check before content is returned.
Request
↓
Authenticate User
↓
Read Metadata
↓
Check Permission
↙ ↘
Allowed Denied
↓ ↓
Content 403This also affects caching. We cannot blindly serve a private paste from the same shared cache used for public content.
Any cache containing private data must preserve the same authorization rules as the main read path.
The service should enforce a maximum paste size.
For example:
Maximum paste size = 1 MBThe exact limit is a product decision, but having one protects the system from unexpectedly large requests that consume too much memory, bandwidth, storage, or cache space.
Users who need to share very large files should use a file storage service instead.
Text usually compresses well, so we can optionally compress paste content before storing or transferring it.
Compression can reduce:
Storage usage
Network bandwidth
Cache usage
However, compression also consumes CPU. Whether it is worth using depends on the average paste size and traffic, so this should be measured rather than assumed.
As the number of pastes grows, we need to make metadata lookups efficient.
Our most common query is:
paste_id → metadataTherefore, paste_id should have a unique index.
Other useful indexes depend on access patterns:
(user_id, created_at) → List a user's pastes
expires_at → Find expired pastesWe should only add indexes that support real queries because every additional index increases storage and write cost.
Eventually, the metadata may become too large for a single database server.
Since most reads already know the paste_id, we can partition the metadata using:
hash(paste_id) → Database ShardFor example:
X7aP9k → Shard 2
K9mQ1a → Shard 5
P2xN8z → Shard 1Hash-based partitioning distributes pastes across shards and works well for direct lookups by paste_id.
Queries such as “show all pastes created by this user” need additional consideration because one user's pastes may be spread across multiple shards. We can handle that with a separate user-oriented index or lookup path if the feature becomes important.
Partitioning helps us distribute data, but we also need to protect it from machine failures.
For the metadata database, each shard can have replicas:
Primary
/ \
Replica A Replica BWrites go to the primary, while some reads can be served from replicas. If the primary fails, a replica can be promoted.
Object storage should also keep redundant copies of paste content across storage nodes or availability zones.
This prevents a single machine or disk failure from losing permanent paste data.
Database replication is not always immediate.
Suppose a user creates a new paste:
Primary → X7aP9k exists
Replica → Not updated yetIf the user immediately reads from the replica, they might incorrectly receive a not found response.
This is a read-after-write consistency problem.
For newly created pastes, we can temporarily route reads to the primary or use a database configuration that provides the required consistency.
Our distributed cache protects the database and object storage, but extremely popular public pastes can still generate heavy traffic to the application servers.
For this case, we can add a CDN:
User
↓
CDN / Edge
├── HIT → Return Paste
└── MISS → Paste ServiceThe CDN caches suitable public content closer to users, which reduces latency and origin traffic.
We should not blindly cache private content at a shared edge. Private pastes require correct authentication, authorization, and cache-control rules.
So the caching strategy becomes:
Normal Paste → Distributed Cache
Viral Public Paste → CDN + Distributed CacheWe introduce the CDN only when traffic patterns justify the additional complexity.

The simplest design treats paste content as immutable.
Once created:
paste_id → fixed contentIf a user wants to change the content, they create a new paste.
Immutability makes caching, CDN delivery, and storage much simpler because the content behind a paste ID does not change.
We still need cache invalidation when a paste is deleted or its access settings change.
For example:
Delete Paste
↓
Mark as Deleted
↓
Invalidate Cache
↓
Remove CDN CopyOtherwise, an old cached copy could remain accessible after deletion.
If editable pastes become a requirement, we would need more careful cache invalidation and possibly content versioning.
A public Pastebin service accepts user-generated content, so it needs protection against abuse.
Anonymous users or bots could create huge numbers of pastes and waste storage.
We can rate limit paste creation using signals such as:
IP address
User account
API key
The exact limits depend on the product and expected traffic.
The service may also receive spam, malicious links, leaked credentials, or other prohibited content.
Common protections include content-size limits, automated abuse detection, user reporting, account restrictions, moderation, and takedown workflows.
Users may paste JavaScript, Python, shell commands, or other code.
The Pastebin service should store and display this code as text, not execute it.
Syntax highlighting only changes how the code is displayed. It should never cause user-submitted code to run on our servers.
Paste content is untrusted user input.
For example, someone could submit:
<script>...</script>When displaying the paste, the service must escape or safely render this content so it appears as text instead of executing in another user's browser.
This protects users from attacks such as cross-site scripting (XSS).
Private pastes need real access control:
Request
↓
Authentication
↓
Authorization
↓
Return ContentThey should also use encrypted network connections and appropriate encryption at rest based on the product's security requirements.
Most importantly, a random or hard-to-guess paste ID is not authorization. The server must verify that the requesting user is allowed to access the paste.
The service should continue working when individual components fail.
Paste Service instances should be stateless so we can run several behind a load balancer:
Load Balancer
/ | \
Server A Server B Server C
DOWN UP UPIf one instance fails, requests can be routed to the remaining healthy instances.
The cache is an optimization, not the source of truth.
If it becomes unavailable, the Paste Service can fall back to the metadata database and object storage.
Cache unavailable
↓
Metadata DB + Object Storage
↓
Return PasteThe system will be slower, and the storage layer may receive a sudden traffic spike. Rate limiting and controlled fallback can help protect it from overload.
A metadata database failure is more serious because metadata tells us whether a paste exists, has expired, is deleted, or requires authorization.
We can reduce this risk with replication, automatic failover, and backups.
Some cached public pastes may still be available during a database outage if our consistency rules allow it. Private content, however, must never be served from cache in a way that bypasses authorization.
If metadata loads quickly but object storage is slow, paste reads can still become slow.
We can reduce the impact using:
Timeouts
Bounded retries
Cache
Storage replication
Retries must be limited. Aggressive retries during a storage outage can create a retry storm and make the failure worse.
Now let's combine the main pieces.
Client
↓
Load Balancer
↓
Paste Service
↓
Rate Limit + Validate Request
↓
Generate Paste ID
↓
Store Content
↓
Store Metadata
↓
Publish Events
↓
Return Paste URLBackground workers can later handle tasks such as:
Search indexing
Expiration cleanup
Analytics
ModerationOnly the data required to create a valid paste should remain on the synchronous request path.
When a user opens a paste:
Request
↓
Paste Service
↓
Check Cache
↓
Load Metadata if Needed
↓
Check Status + Expiration + Permission
↓
Load Content
↓
Cache Safe Response
↓
Return PasteA view event can be published asynchronously for analytics.
The important point is that the service checks whether the paste is still valid and accessible before returning its content.
Search and analytics remain separate asynchronous systems, so they can scale independently without making the core read and write paths more complex.
As the system grows, the main bottlenecks are:
Hot pastes: Protect storage with distributed caching and CDN caching for suitable public content.
Metadata growth: Use indexes, replicas, and eventually partitioning by paste_id.
Expiration cleanup: Process expired pastes in batches or time buckets instead of scanning the entire database.
Search and analytics: Keep these workloads asynchronous and separate from the core database.
Most of these problems come from either uneven read traffic or growing data volume.
Storing everything in a database is simpler at small scale.
Separating metadata and content adds complexity but allows both storage layers to scale independently.
Random IDs are simple and harder to enumerate, but collisions must be detected and retried.
Distributed IDs guarantee uniqueness before encoding but require an additional ID-generation mechanism.
The service should stop serving deleted or expired pastes immediately.
Physical cleanup can usually happen asynchronously, depending on the product's privacy and retention requirements.
The database and object storage remain the source of truth, while the cache improves latency and protects them from repeated reads.
Updating view counts synchronously gives fresher numbers but creates additional write pressure.
Asynchronous analytics scales better at the cost of slightly delayed counts.
A viral public paste should be served from the highest available cache layer:
Users
↓
CDN / Edge Cache
↓
Distributed Cache
↓
Paste Service
↓
StorageMost requests should be handled by the CDN or distributed cache. Only cache misses should reach the deeper storage layers.
At this scale, we would rely more heavily on:
Partitioned Metadata DB
Distributed Object Storage
Replication
Distributed Cache
Background Workers
Separate Search InfrastructureSince direct lookup by paste_id is the main access pattern, hash(paste_id) remains a natural way to distribute metadata across shards.
If most content expires quickly, expiration becomes a more important part of the architecture.
We can rely more heavily on expiration indexes, delayed queues or time buckets, short-lived cache entries, and background cleanup workers.
This shows an important system-design principle: the architecture should follow the product's actual access and retention patterns.
Putting everything together, our final Pastebin architecture looks like this:
Users
│
▼
┌─────────────┐
│ CDN / Edge │
│ (Public) │
└──────┬──────┘
│
▼
┌─────────────┐
│Load Balancer│
└──────┬──────┘
│
▼
┌─────────────┐
│Paste Service│
└──┬──┬──┬───┘
│ │ │
┌─────────┘ │ └──────────┐
▼ ▼ ▼
┌────────────┐ ┌───────────┐ ┌────────────┐
│Distributed │ │Metadata DB│ │ Object │
│ Cache │ │+ Replicas │ │ Storage │
└────────────┘ └───────────┘ └────────────┘
│
Paste Service ─────────┼──────► Event Queue
│
┌─────────┼─────────┐
▼ ▼ ▼
Expiration Search Analytics
Workers Worker Workers
│ │ │
▼ ▼ ▼
Cleanup Search Analytics
Index StoreThe responsibilities are now clearly separated:
Paste Service: Handles creation, reads, authorization, expiration checks, deletion, and rate limiting.
Distributed Cache: Serves frequently requested pastes without repeatedly accessing storage.
Metadata Database: Stores paste metadata and remains the source of truth for visibility, status, and expiration.
Object Storage: Stores the actual paste content.
CDN: Handles extremely popular public content closer to users.
Event Queue and Workers: Handle expiration cleanup, search indexing, analytics, and other asynchronous work.
This architecture keeps the core create and read paths simple while allowing the expensive parts of the system to scale independently.

Use random Base62 IDs with a uniqueness check and retry on collision. A distributed ID generator with Base62 encoding is another option.
For a small system, the database may be enough. At larger scale, keep metadata in the database and the actual paste content in object storage.
Check expires_at during reads for immediate logical expiration, then delete the data asynchronously using background workers.
Serve popular public content from a distributed cache and, when needed, a CDN so most requests never reach the database or object storage.
Authenticate the requester and perform an authorization check before returning the content. An unlisted or hard-to-guess URL is not a replacement for authorization.
Process both asynchronously. Public searchable pastes can be added to a search index, while view events can be aggregated in an analytics system.
Partition by hash(paste_id) because direct lookup by paste ID is the main access pattern.
Fall back to the metadata database and object storage while protecting them from the sudden increase in traffic.