Durgesh Tiwari
Author
A Chat Application looks simple from the user's side.
You open an app, type a message, tap Send, and the other person receives it almost instantly.
But behind that simple action, many things are happening.
The system needs to know:
Who sent the message?
Who should receive it?
Is the receiver online?
Which server is the receiver connected to?
Should the message be stored first?
How do we keep messages in the correct order?
What happens if the receiver is offline?
What happens if a server crashes?
How do we support group chats with thousands of members?
How do we deliver messages across millions of persistent connections?
These questions make Chat Application System Design one of the most useful topics for learning real-time distributed systems.
A Chat Application is a communication system that allows users to exchange messages in near real time.
Common features include:
One-to-one messaging
Group chat
Text messages
Images and videos
Files
Voice messages
Online status
Typing indicators
Delivery receipts
Read receipts
Message history
Push notifications
A simple flow looks like:
User A
↓
Chat Server
↓
User BFor a few users, this is easy.
At millions of users and persistent connections, the architecture becomes much more complex.
Traditional web applications often use short request-response communication:
Client → Request → Server → ResponseChat applications need something different.
Users expect messages to arrive without refreshing or repeatedly asking the server for updates.
A large chat system must handle:
Real-Time Delivery
Persistent Connections
Message Durability
Low Latency
Message Ordering
Offline Users
Presence Tracking
Group Fan-Out
High Availability
Massive ScaleThe system must solve real-time delivery and durable storage at the same time.
In a system design interview, start with requirements instead of immediately naming technologies such as Kafka, Redis, or Cassandra.
A user should be able to send a private message to another user.
Alice → "Hi Bob"
Bob → "Hey Alice"If Bob is online, the message should reach him quickly.
If Bob is offline, the message should remain stored and become available later.
Users should be able to communicate inside groups.
Study Group
├── Alice
├── Bob
├── John
├── Priya
└── RahulOne group message may need to reach many recipients.
This introduces the fan-out problem.
Users should be able to open a conversation and retrieve previous messages.
Messages therefore require durable storage.
The application may show states such as:
SENT
DELIVERED
READThe exact meaning of each state should be clearly defined.
Users may see:
Online
Offline
Last SeenThe system therefore needs presence information.
Typing indicators are temporary events:
Alice is typing...They generally do not need permanent database storage.
The system may support:
Images
Videos
Audio
PDFs
Documents
Large media should normally be stored in object storage, not inside the primary message database.
Important non-functional requirements include:
Low latency
High availability
Scalability
Reliability
Durability
Correct message ordering
Chat is interactive.
Messages should normally reach online recipients quickly.
If a chat server fails, users connected to it should reconnect to another healthy server.
Avoid single points of failure.
A message should not disappear because a server crashes after acknowledging it.
The system should make important messages durable before considering them safely accepted.
Suppose the system has:
500 million registered users
100 million daily active users
20 million concurrent usersHandling millions of long-lived connections requires horizontal scaling across many servers.
Messages inside a conversation should appear in the intended order.
1. I reached the airport.
2. Boarding now.
3. See you soon.Conversation-level ordering is normally more important than global ordering across the entire platform.
Suppose:
100 million daily active users
50 messages per user per day
Messages per day:
100,000,000 × 50
= 5 billion messages/dayAverage message rate:
5,000,000,000 / 86,400
≈ 58,000 messages/secondPeak traffic may be several times higher.
We may also have millions of concurrent WebSocket connections.
This immediately tells us:
One server is not enough.
Message storage must handle very high write volume.
Connection state must be distributed.
Message delivery must work across servers.
A scalable design may begin with:
Mobile / Web Clients
↓
API / WS Gateway
↓
┌─────────────┴─────────────┐
↓ ↓
Chat Server A Chat Server B
└─────────────┬─────────────┘
↓
Message Service
/ \
↓ ↓
Message Store Event Broker
↓
Other Chat ServersAdditional services can later include:
Presence Service
Notification Service
User Service
Group Service
Media Service
Cache
With polling:
Client → Any new message?
Server → No
Client → Any new message?
Server → No
Client → Any new message?
Server → YesThis creates repeated requests even when no message exists.
It can waste network and server resources at scale.
With long polling, the server holds the request until new data becomes available.
Client ───── Request ─────→ Server
│
│ waits
│
New Message ────────────────┘
│
Client ←──── Response ──────┘Long polling works, but the client usually opens another request after every response.
A WebSocket creates a long-lived, two-way connection.
Client ⇄ WebSocket ⇄ Chat ServerBoth sides can send events whenever necessary.
WebSockets are well suited for:
New messages
Typing indicators
Delivery receipts
Read receipts
Presence events
A practical system may use:
HTTP / REST
→ Login
→ Profiles
→ Message history
→ Media APIs
WebSocket
→ Messages
→ Typing
→ Presence
→ Delivery/read eventsSuppose Alice opens the application.
After authentication, her client establishes a WebSocket connection.
Alice's Device
↓
WebSocket
↓
Chat Server 7The system now needs to know:
Alice → Chat Server 7Otherwise, when another user sends Alice a message, the system will not know where her active connection lives.
With many servers, we need mappings such as:
Alice → Server 7
Bob → Server 15
John → Server 2
Priya → Server 28For multi-device users, one user may have several mappings.
A distributed cache such as Redis can be useful for this short-lived connection state.
Example:
user_id → active connection/server informationThe data is temporary and should disappear or expire after disconnection.
Suppose Alice sends:
Hello Bob!A common logical flow is:
Alice
↓
Chat Server A
↓
Message Service
├────────→ Message Database
↓
Find Bob's Connection
↓
Routing / Broker
↓
Chat Server B
↓
BobExample payload:
{
"conversation_id": "conv_101",
"client_message_id": "abc-123",
"text": "Hello Bob!"
}The server checks:
Authentication
Conversation membership
Message size
Rate limits
Authorization
The service may create:
message_id
conversation_id
sender_id
content
created_at
sequence_numberThe message should be written to durable storage.
Message Service
↓
Message DatabaseThe connection or presence store may return:
Bob → Chat Server BChat Server B forwards the message through Bob's WebSocket.
Chat Server B
↓
Bob's Device
Real-time delivery and durable storage should be treated as separate concerns.
For an online user:
Store Message
+
WebSocket DeliveryFor an offline user:
Store Message
+
Optional Push Notification
+
Synchronize After ReconnectWhen Bob reconnects, he can request messages after the last message he received.
For example:
GET /conversations/101/messages?after=msg_900000The database remains the durable source of message history.
WebSocket delivery is the real-time delivery mechanism.
A simplified model may contain:
User
----------------
user_id
name
profile_photo
created_atConversation
----------------
conversation_id
conversation_type
created_atconversation_type may be:
DIRECT
GROUPConversationMember
---------------------
conversation_id
user_id
joined_at
role
last_read_sequenceMessage
----------------------
message_id
conversation_id
sender_id
message_type
content
created_at
sequence_numberExample:
message_id: msg_9001
conversation_id: conv_500
sender_id: user_12
message_type: TEXT
content: "Meeting starts at 5."
sequence_number: 109conversation_id is important because message history is usually read conversation by conversation.
There is no universally best database for chat.
The correct choice depends on scale and access patterns.
For smaller or moderate systems, databases such as PostgreSQL or MySQL can work well.
Benefits include:
Transactions
Strong consistency
Mature indexes
Relational modeling
Easy development
At very large scale, messages become:
Extremely write-heavy
Append-oriented
Very large in volume
Commonly queried by conversation
A distributed NoSQL or wide-column-style store may become attractive.
The key reasoning is:
Messages are mostly appended and later read by conversation in time order, so storage should support high write throughput and efficient ordered range reads by conversation.
Choose storage based on access patterns, not popularity.
At billions of messages, the dataset must be partitioned.
A natural partition key is:
conversation_idbecause typical queries look like:
Give me messages from conversation 500.Messages belonging to the same conversation can be colocated for efficient history reads.
Example:
Shard 1
├── Conversation 1
├── Conversation 5
└── Conversation 9
Shard 2
├── Conversation 2
├── Conversation 6
└── Conversation 10Partitioning by user_id can become awkward for group messages because one conversation belongs to many users.
conversation_id often maps more naturally to message-history queries.
However, extremely large conversations may create hot partitions.
Large channels may therefore require additional partitioning strategies.
Network timestamps alone may not provide reliable ordering because clocks differ and messages may be processed concurrently.
A common approach is to maintain an ordering value within each conversation.
Conversation 100
Message A → sequence 501
Message B → sequence 502
Message C → sequence 503Clients display messages according to this sequence.
Usually, no.
Global ordering across every conversation would add unnecessary complexity.
What normally matters is:
Ordering within a conversationThis is an important system design trade-off.
Message IDs must be unique.
Possible approaches include:
UUIDs
Time-based distributed IDs
Snowflake-style IDs
Unique identification and conversation ordering should be treated as separate concerns unless the chosen ID-generation strategy explicitly guarantees both.
Unique Message ID
+
Conversation OrderingMobile networks are unreliable.
Suppose the server stores Alice's message but Alice loses connectivity before receiving the acknowledgement.
Her client retries.
Without protection:
Hello
Hellomay appear twice.
A common solution is a client-generated idempotency key:
client_message_id = abc-123First request:
abc-123
→ Store MessageRetry:
abc-123
→ Duplicate Detected
→ Return Existing MessageThis makes the send operation idempotent.
A chat system may have multiple stages:
Sender Sends
↓
Server Accepts
↓
Recipient Device Receives
↓
Recipient ReadsThese may correspond to:
SENT
DELIVERED
READDefinitions must be clear.
For example:
SENT → server accepted and durably stored the message.
DELIVERED → recipient device acknowledged receipt.
READ → recipient viewed or acknowledged the conversation.
Instead of storing one read row for every user-message combination, track progress per conversation.
For example:
Bob
conversation_id = 100
last_read_sequence = 503Messages with sequence numbers up to 503 are considered read by Bob.
This is significantly more efficient for large histories.
Typing indicators are ephemeral events.
Alice
↓
WebSocket Server
↓
Typing Event
↓
BobThey usually:
Are not persisted permanently
Expire after a few seconds
Are delivered only to currently relevant users
This separates temporary real-time events from durable messages.
Presence tracks whether users are currently active.
Alice → ONLINE
Bob → OFFLINE
John → ONLINEAn active WebSocket suggests a user is online.
However, mobile networks may disappear without a clean disconnect.
Therefore, presence should often use heartbeats and expiration.
Example:
Client → PING
Server → PONGIf several heartbeats are missed, the server can consider the connection stale.
Presence records may use a TTL so they expire automatically.
The system may also maintain:
last_seen_atPresence is frequently changing data, so a fast distributed store or cache is often appropriate.
Writing every heartbeat to a permanent relational database would usually be unnecessary.
Suppose Alice sends a message to a group.
Group Message
↓
┌────────┼────────┐
↓ ↓ ↓
Bob John PriyaThis distribution process is called fan-out.
When the message is created, delivery work is generated for recipients.
Advantages:
Fast recipient-side reads
Delivery information can be prepared early
Disadvantages:
High write amplification
Very expensive for huge groups
Store one copy of the message.
Users retrieve it when they read the conversation.
Advantages:
Lower write amplification
Better suited to extremely large channels
Disadvantages:
More work may happen during reads
Situation | Common Strategy |
|---|---|
Small groups | Fan-out on write can work well |
Very large groups | Fan-out on read may be better |
Mixed workloads | Hybrid strategy |
For a group with one million members, blindly generating one million durable copies per message may be prohibitively expensive.
A large-group design may:
Store Message Once
↓
Update Group Stream
↓
Push to Active Subscribers
+
Offline Users Fetch Later
Suppose:
Alice → Server 1
Bob → Server 8Server 1 needs a way to route events toward Server 8.
A broker or event-streaming system can help.
Server 1
↓
Message Broker
↓
Server 8A distributed broker may provide:
Buffering
Decoupling
Partitioning
Replay
Fault tolerance
Kafka or similar technologies may be useful depending on scale.
No.
For a smaller product:
WebSocket Servers
↓
Database
+
Redismay be enough.
A distributed broker becomes more useful when the system has:
Many servers
Very high event volume
Asynchronous consumers
Cross-server delivery
Analytics pipelines
Multiple downstream services
Architecture should match requirements.
Do not load the entire conversation history at once.
Use pagination.
For example:
GET /conversations/123/messages
?before=msg_9000
&limit=50Return the latest messages, then load older messages as the user scrolls.
This improves:
Database performance
Network usage
Client performance
User experience
Cursor-based pagination is generally preferable to deep offset pagination for large message histories.
Large media should not be stored inside normal message rows.
A better flow is:
Client
↓
Media Upload Service
↓
Object Storage
↓
Return Media ID
↓
Send Chat MessageThe message can contain metadata:
{
"message_type": "IMAGE",
"media_id": "media_8821",
"thumbnail": "..."
}The actual media remains in object storage.
A CDN can be used to serve frequently accessed media efficiently.
Do not push a 20 MB video through a WebSocket chat server.
Instead:
Client
↓
Request Upload URL
↓
Media Service
↓
Presigned Upload URL
↓
Client
↓
Object StorageAfter upload, the client sends only the media reference through the messaging system.
If Bob is offline, the WebSocket path is unavailable.
The Chat Service can generate a notification event:
Chat Service
↓
Notification Service
↓
Push Provider
↓
Bob's DevicePush notifications are only a notification mechanism.
They should not become the durable message store.
When Bob opens the app, the real message should come from message storage.
A user may be active on several devices:
Bob
├── Phone → Server 4
├── Laptop → Server 7
└── Web → Server 12A new message may need to be delivered to all appropriate active sessions.
Read state also needs synchronization.
For example:
Bob Phone
↓
last_read_sequence = 900
↓
Chat Service
├──→ Bob Laptop
└──→ Bob WebSuppose Chat Server 8 crashes.
All WebSocket connections on that server disappear.
Clients should reconnect through the load balancer.
Client
↓
Connection Lost
↓
Load Balancer
↓
Healthy Chat ServerAfter reconnecting, the client can send its synchronization position:
last_received_sequence = 901The server returns any missing messages.
This is another reason durable message storage is essential.
Thousands or millions of clients should not reconnect at exactly the same time.
Use exponential backoff with jitter.
For example:
Retry after ~1 second
Retry after ~2 seconds
Retry after ~5 seconds
Retry after ~10 secondsJitter randomizes retries and helps prevent a reconnect storm after an outage.
WebSocket connections may remain open for hours.
Therefore, scaling based only on HTTP request count is insufficient.
Useful signals may include:
Active connection count
Memory
CPU
Network bandwidth
Event rate
Suppose:
Server A → 100,000 connections
Server B → 10,000 connectionsThe system should avoid continuing to overload Server A simply because CPU usage is still similar.
Sticky sessions can simplify some designs by returning a client to the same server.
However, the architecture should not depend on them for correctness.
When a server fails, the user must reconnect elsewhere.
Important connection state should therefore be recoverable through distributed infrastructure rather than living only inside one server.
A malicious client should not be able to overwhelm the chat platform.
Rate limiting may be applied by:
User
IP address
Conversation
Tenant
API key
Example:
Allowed:
100 messages/minute
Exceeded:
429 Too Many RequestsReal systems may also require:
Spam detection
Block lists
User reports
URL scanning
Media scanning
Abuse classifiers
Fraud detection
Some of these tasks can happen asynchronously so normal message delivery remains fast.
Chat applications handle private user communication.
Important security controls include:
TLS
Authentication
Authorization
Secure session/token management
Encryption at rest
Access control
Audit logging
Rate limiting
Abuse protection
Before returning conversation history, always verify that the requesting user belongs to the conversation.
Authorization mistakes can expose private messages.
Some chat systems implement end-to-end encryption (E2EE).
Alice Encrypts
↓
Server Stores / Routes Ciphertext
↓
Bob Receives Ciphertext
↓
Bob DecryptsIdeally, the server cannot read the plaintext.
However, E2EE significantly changes the architecture.
Challenges include:
Key management
Device provisioning
Multi-device synchronization
Group membership changes
Backup and recovery
Search
Moderation
Message restoration
E2EE should not be treated as a simple checkbox feature.
At smaller scale, database search may be sufficient.
At larger scale, use a dedicated search index.
Message Service
↓
Message Database
└────→ Search IndexSearch indexing can be asynchronous because small delays before a new message becomes searchable are often acceptable.
A distributed cache can help with:
User profiles
Group membership
Presence
Conversation metadata
Recent messages
However, the cache should generally remain an optimization, not the durable source of chat history.
Instead of scanning every message, maintain progress values such as:
latest_conversation_sequence
last_read_sequenceA simplified unread estimate becomes:
Unread ≈ latest sequence - last read sequenceThe exact implementation may need adjustments for deleted messages, system events, or membership changes.
Hide the message only for one user.
A tombstone may be stored:
message_id
deleted = true
deleted_atTombstones can help synchronize deletion across devices.
Edited messages may maintain:
content
edited_at
versionFor audit-sensitive systems, older versions may also be retained.
Connected clients receive an update event.
Offline clients see the latest state after synchronization.
Mobile / Web Clients
↓
API / WS Gateway
↓
┌───────────┴───────────┐
↓ ↓
Chat Server A Chat Server B
└───────────┬───────────┘
↓
Message Service
┌──────────┼───────────┐
↓ ↓ ↓
Message DB Broker Cache
│
┌───────────────┼───────────────┐
↓ ↓ ↓
Connection / Presence Notification
Routing Service Service
Additional Services
───────────────────
User Service
Group Service
Media Service
Object Storage
Search
MonitoringEach component exists to solve a specific scaling or product requirement.
Suppose Alice sends Bob:
"Are you available at 4 PM?"The complete lifecycle is:
1. Alice has an active WebSocket connection
↓
2. Client creates client_message_id
↓
3. Message sent to Chat Server
↓
4. Authentication + authorization + validation
↓
5. Message Service assigns message_id/order
↓
6. Message stored durably
↓
7. Alice receives server acknowledgement
↓
8. Connection service locates Bob
↓
9. Event routed to Bob's Chat Server
↓
10. Bob receives message
↓
11. Bob's device acknowledges delivery
↓
12. Bob reads conversation
↓
13. Read progress updated
↓
14. Alice receives read-status eventFeature | HTTP Polling | WebSocket |
|---|---|---|
Connection | Repeated requests | Long-lived connection |
Server Push | Not natural | Supported |
Latency | Depends on polling interval | Low |
Network Overhead | Higher | Lower for frequent events |
Complexity | Simpler | More connection management |
Chat Suitability | Limited at scale | Strong choice for real-time chat |
Feature | SQL | Distributed NoSQL |
|---|---|---|
Transactions | Strong support | Depends on system |
Relational Modeling | Strong | Usually limited |
Horizontal Write Scale | Can require more work | Often designed for it |
Large Message Dataset | Works at moderate scale | Useful at very large scale |
Best Choice | Depends on workload | Depends on workload |
Feature | Fan-Out on Write | Fan-Out on Read |
|---|---|---|
Write Cost | Higher | Lower |
Read Cost | Lower | Higher |
Small Groups | Good fit | Possible |
Massive Groups | Expensive | Often better |
Main Trade-Off | Write amplification | More work during reads |
Stronger guarantees may be more important for:
Message persistence
Conversation ordering
Membership authorization
Eventual consistency may be acceptable for:
Presence
Typing indicators
Search indexing
Some unread counters
Not every feature requires the same consistency level.
HLD focuses on:
WebSocket infrastructure
Chat servers
Message Service
Message storage
Brokers
Presence
Notifications
Media storage
Partitioning
Scaling
Failure handling
It answers:
How does the chat system work at scale?
LLD focuses on application classes and interfaces such as:
User
Conversation
Message
Group
Membership
MessageRepository
ChatService
DeliveryService
PresenceServiceMessage types may also use structures such as:
Message
├── TextMessage
├── ImageMessage
├── VideoMessage
└── FileMessageLLD answers:
How should the software components and objects be structured?
Do not start with the most complex architecture.
Clients
↓
Chat Server
↓
SQL DatabaseAdd:
WebSockets
Connection TrackingAdd:
Multiple Chat Servers
Load Balancer
Distributed CacheAdd:
Message Broker
Database Partitioning
Presence Service
Media Service
Notification ServiceAdd:
Multi-Region Deployment
Cross-Region Routing
Advanced Partitioning
Large-Group Optimization
Advanced MonitoringBuild complexity only when requirements justify it.
Use WebSockets for real-time communication and normal HTTP APIs for operations such as login, profiles, message history, and media workflows.
Multiple chat servers maintain active connections.
A Message Service validates and durably stores messages before routing them to online recipients.
Offline users retrieve missed messages after reconnecting and may receive push notifications.
WebSockets provide a persistent, bidirectional connection.
This allows the server to push messages immediately without repeated polling.
They work well for messages, typing indicators, presence events, and delivery receipts.
Both can work.
Long polling uses repeated HTTP requests and is simpler in some environments.
WebSockets provide continuous two-way communication and are generally better suited to highly interactive chat systems at large scale.
Partition message data across database nodes.
A useful partition key is often conversation_id because message-history queries usually operate within one conversation.
Use efficient pagination and keep large media in object storage.
Maintain an ordering value such as a sequence number within each conversation.
Global ordering across every conversation is normally unnecessary.
Store messages durably regardless of receiver presence.
For offline users, optionally send a push notification.
When they reconnect, synchronize messages after their last received sequence or cursor.
Use a client-generated client_message_id or idempotency key.
When the client retries, the server detects the previous request and returns the existing message instead of inserting another copy.
Maintain distributed mappings between users and active connections.
Use heartbeats and TTL expiration to remove dead sessions.
A fast distributed cache is commonly suitable.
Store group membership separately.
For small groups, fan-out on write may work well.
For very large groups, fan-out on read or a hybrid approach can reduce write amplification.
Distribute connections across many servers using a load balancer.
Track connection locations in a distributed service.
Scale horizontally according to connection count, memory, network usage, and event rate.
Connected clients lose their connections and reconnect to another healthy server.
Because messages are stored durably, they can synchronize anything missed during the failure.
There is no universal answer.
SQL can be suitable at moderate scale.
At very high message volumes, distributed storage may offer better horizontal write scalability.
The choice should depend on access patterns, scale, consistency, and operational requirements.
Large media should generally live in object storage.
The message database stores only metadata or a media reference.
This reduces database size and makes CDN-based delivery easier.
Store the highest conversation sequence read by each user.
For example:
Bob
last_read_sequence = 850Messages through sequence 850 can be treated as read.
Typing indicators are short-lived WebSocket events.
They usually are not permanently stored.
They should expire automatically after a short period.
Use:
Multiple WebSocket servers
Load balancing
Distributed connection tracking
Horizontal scaling
Efficient cross-server routing
Connection-aware metrics
Monitor connection count, memory, CPU, bandwidth, and event rate.
Avoid creating one million durable message copies unless required.
Store the group message once and use fan-out-on-read or a hybrid approach.
Push immediately to active subscribers and allow offline users to retrieve the message later.
Not necessarily.
A broker is useful when there are many chat servers, very high event volume, multiple asynchronous consumers, or cross-service event processing.
A smaller application may not need the extra complexity.
First make the message durable.
Then attempt real-time delivery.
Use acknowledgements, retry mechanisms, synchronization, and idempotency.
Exactly-once end-to-end delivery is difficult, so practical systems usually provide reliable delivery plus duplicate protection.
Track all active sessions for the user.
Send new events to each required device.
Persist shared conversation state such as read progress and synchronize it across devices.
Encrypt messages on the sender's device and decrypt them only on authorized recipient devices.
Servers store and forward ciphertext.
The difficult areas are key management, multi-device synchronization, group membership changes, backup, and recovery.
Use persistent connections, efficient event routing, fast connection lookup, well-partitioned storage, caching, and geographically appropriate infrastructure.
Avoid unnecessary synchronous dependencies in the critical send path.
Measure sender-to-recipient latency, not only API response time.
Common mistakes include:
Designing only online delivery and ignoring offline users
Storing messages only inside WebSocket servers
Trying to guarantee unnecessary global ordering
Ignoring duplicate messages
Ignoring reconnect synchronization
Storing large media in the primary message database
Keeping presence state on only one server
Ignoring large-group fan-out
Adding Kafka without explaining the requirement
Ignoring multi-device synchronization
Ignoring authorization
Treating push notifications as durable message storage
Depending on sticky sessions for correctness
A good architecture solves the requirements with the least unnecessary complexity.