chatapp

Starting Point

Starting point: The simplest possible chat system. A single server holds all WebSocket connections. When Client A sends a message, the server looks up Client B's connection in memory and forwards the message directly.

How it works:

1. Both clients establish WebSocket connections to the Chat Server.
2. Client A sends send_message with receiver's user ID.
3. The server looks up Client B's WebSocket in its in-memory connection map.
4. If found: forward the message directly. If not found: write to database and hope Client B fetches it later.

Three critical flaws:

We need to solve all three. Let's start with the most fundamental: making the system real-time and multi-server.

Scaling Connections

Problem being solved: A single server can't hold 10M connections. We need a fleet of ~10 Chat Servers (each handling ~1M connections). But now: when Client A sends a message to Client B, how does the system know which Chat Server holds Client B's connection?

Solution: Add a Connection Registry — a shared Redis store that maps every connected user to their Chat Server.

Message flow:

1. Client A sends send_message over WebSocket to Chat Server 1.
2. Chat Server 1 queries the Connection Registry (Redis): HGET connections:{receiver_id} server_id → returns chat-server-2.
3. Chat Server 1 forwards the message directly to Chat Server 2 via internal RPC.
4. Chat Server 2 delivers incoming_message to Client B over WebSocket.
5. Chat Server 1 writes the message to the Database.

The Connection Registry is a Redis hash map:

Key: connections:{user_id}
Value: {server_id: "chat-server-2", connected_at: 1710684600000}

• On connect: Chat Server writes HSET connections:{user_id} server_id chat-server-2.
• On disconnect: Chat Server deletes the entry.
• TTL-based cleanup: Each entry has a 60-second TTL, refreshed by periodic heartbeats. If a Chat Server crashes, stale entries expire automatically.

What we've solved:

• ✅ Scale: 10 servers × 1M connections each = 10M concurrent connections.
• ✅ Routing: Connection Registry lets any server find any user's connection.

What's still broken:

• ❌ Durability: If Chat Server 1 crashes after forwarding to Server 2 but before writing to DB → the message was delivered but not persisted (no history, no audit trail). If Server 2 crashes before delivering → message is lost.
• ❌ Offline delivery: If Client B isn't connected, we write to DB... but how does Client B know to check? When they come online, there's no mechanism to deliver stored messages.
• ❌ Tight coupling: The sending Chat Server must know about the receiving server, the database, and every other downstream concern.

Guaranteed Delivery

Problem being solved: In the direct server-to-server model, a crash at any point can lose the message. The sender gets no reliable acknowledgment. We need the message to be durably stored before we tell the sender "your message is safe".

Solution: Insert Kafka (message queue) between the Chat Server and the delivery pipeline. The Chat Server's only job is: receive the message → enqueue to Kafka → ACK to sender. A separate MQ Consumer handles delivery asynchronously.

Send path (4 steps, all synchronous to the sender):

1. Client A sends send_message over the WebSocket to Chat Server 1.
2. Chat Server 1 requests a unique message ID from the ID Generator (Snowflake).
3. Chat Server 1 writes the message into Kafka (Message Queue).
4. Once Kafka acknowledges, Chat Server 1 sends message_received (✓) back to Client A. At this point, the message is durably stored in Kafka — it will not be lost even if every Chat Server crashes simultaneously.

Delivery path (asynchronous, handled by the MQ Consumer):
5. The MQ Consumer reads the message from Kafka.
6. MQ Consumer queries the Connection Registry to find Client B's Chat Server.
7. Message is routed to Chat Server 2 → delivered to Client B → Client B confirms → stored in Message DB → sender notified (✓✓).

Why Kafka changes everything:

What we've solved:

• ✅ Durability: The ✓ checkmark now means "your message is safe" — not just "I received it".
• ✅ Decoupling: Chat Server doesn't know about the delivery pipeline, DB, or inbox.

What's still broken:

• ❌ Offline delivery: If Client B is offline, the MQ Consumer can write to the DB... but when Client B comes online, how does it know there are messages waiting?

Offline Message Handling

Problem being solved: When the MQ Consumer tries to deliver a message and Client B is offline (no entry in the Connection Registry), or if the delivery confirmation (message_delivered) doesn't arrive within 10 seconds, the message has nowhere to go.

Solution: Add an Inbox Cache — a per-user Redis Sorted Set (ZSET) that stores undelivered messages, sorted by timestamp. When Client B comes online, the system drains the inbox before switching to real-time delivery.

Offline delivery flow (message arrives while Client B is offline):

1. MQ Consumer reads message from Kafka.
2. Queries Connection Registry → Client B has no active connection.
3. Writes message to Message DB with undelivered: true.
4. Pushes message to Client B's Inbox Cache: ZADD inbox:{user_B} 1710684600000 "{msg_json}".
5. Sends push notification via APNs/FCM to wake Client B's device.

Reconnection flow (Client B comes online):

1. Client B connects and establishes a WebSocket with Chat Server.
2. Chat Server registers the connection in the Connection Registry.
3. Chat Server queries the Inbox Cache: ZRANGE inbox:{user_B} 0 -1 (oldest first).
4. For each undelivered message, Chat Server sends incoming_message to Client B.
5. As Client B confirms each message (message_delivered), the server:
   • Updates the Message DB: undelivered → false.
   • Removes the message from the Inbox Cache: ZREM inbox:{user_B} message.
6. Once the inbox is drained, Client B is in sync and receives new messages in real time.

ZADD inbox:{user_id} 1710684600000 "{msg_json_1}"
ZADD inbox:{user_id} 1710684601000 "{msg_json_2}"
ZPOPMIN inbox:{user_id}  // returns msg_json_1 (oldest first)

What we've solved:

• ✅ Offline delivery: No message is lost. Inbox stores everything until Client B confirms receipt.
• ✅ Ordering: ZSET sorts by timestamp. Messages drain oldest-first.

What's still broken:

• ❌ Duplicates: If Client B's message_delivered ACK is lost, the server retries delivery — potentially showing the same message twice. We need exactly-once semantics.

Final Design

The last piece: deduplication for exactly-once delivery. We combine at-least-once delivery (guaranteed by Kafka + inbox + retries) with idempotency checks to suppress duplicates.

Exactly-Once Delivery: Three Dedup Layers

Duplicate messages can occur when:

• Client retries a send_message that the server already processed (network timeout).
• The MQ Consumer delivers a message, but the delivery ACK is lost, so it retries.
• The inbox drain sends a message to Client B, but the ACK is lost.

Prevention mechanisms:

NFR Scorecard — All Requirements Met

Deep Dive #1

The storage model for a chat app is unique: most messages are ephemeral — stored on the server only until delivered, then deleted. This is fundamentally different from social media (store forever) or email (store until user deletes).

Database choice: Hybrid SQL + NoSQL

• MongoDB for messages — high write throughput, flexible schema, good for time-series-like queries (messages by conversation, ordered by timestamp).
• PostgreSQL for user profiles, settings, and relationships — relational queries, ACID transactions for user management.
• Redis for caches — connection registry, inbox ZSET, presence status.

Deep Dive #2

Group chat multiplies the delivery problem: a single message must be delivered to every member of the group. The approach depends on group size.

Small groups (≤ 500 members) — push model:

When a user sends a message to a group:

1. The message is published to Kafka.
2. The MQ Consumer fetches the group member list from the database.
3. For each member, it follows the same delivery pipeline as 1:1 messages: check connection registry → deliver via WebSocket if online → store in inbox if offline.
4. The message is stored once in the Message DB (with group_id instead of receiver_id).

This is essentially fan-out-on-write for groups.

Large groups (> 500 members) — pull model:

For large groups (Slack channels, Discord servers with 10K+ members), pushing to every member is too expensive. Instead:

1. The message is stored in the Message DB under the group.
2. Cached in Redis: ZADD group_messages:{group_id} timestamp message_json.
3. Clients poll periodically: GET /api/chat/{group_id}?timestamp={last_seen}.
4. The server returns new messages since last_seen from the Redis cache.

Deep Dive #3

Users expect to see whether their contacts are online, offline, or "last seen 5 minutes ago". At 100M DAU with 10M concurrent connections, the presence system generates a massive volume of status change events.

Heartbeat-based presence:

Each connected client sends a heartbeat to the Chat Server every 30 seconds. The Chat Server updates the user's presence in Redis:

SET presence:{user_id} {timestamp} EX 60

The 60-second TTL acts as a natural "offline" detector. If the client stops sending heartbeats (app backgrounded, network lost), the key expires and the user is considered offline.

Querying presence:

When Client A opens a chat with Client B, the app queries: GET presence:{user_b_id}.

• If the key exists: user is online (or was active within 60 seconds). Display "Online" or "Last seen {timestamp}".
• If the key doesn't exist: user is offline.

Deep Dive #4

WhatsApp is famous for end-to-end encryption (E2EE): the server never sees the plaintext content of messages. This has significant architectural implications.

How E2EE works (Signal Protocol):

1. Each user generates a key pair (public + private) on their device at registration.
2. The public key is uploaded to the server. The private key never leaves the device.
3. When Alice wants to message Bob: Alice fetches Bob's public key from the server.
4. Alice encrypts the message with Bob's public key.
5. The encrypted message traverses the server (stored in MongoDB, queued in Kafka, etc.) — but the server cannot decrypt it.
6. Bob receives the encrypted message and decrypts it with his private key.

These open-ended topics test architectural judgment at the staff+ level.