You are building a live sports score page. How do clients get the updates, and why that choice?

Scores flow one way, from the server to many viewers, so Server-Sent Events are a good fit: a simple stream over normal HTTP with automatic reconnect and a way to resume from the last event. WebSockets also work but add a two-way channel nobody needs. Short polling every few seconds is the simplest and is fine if a few seconds of delay is acceptable, and it caches well at a CDN. The trade-off is delay and wasted requests with polling, against the cost of holding one open connection per viewer with push.

A server holding 100,000 WebSocket connections restarts. What goes wrong, and how do you prevent it?

All 100,000 clients notice the drop and reconnect at once, a thundering herd that can overload the remaining servers and the database behind them, and may knock them over too. Prevent it with exponential backoff plus random jitter in the client, so retries spread over time, with connection limits and rate limits on the server side, and by draining a server gradually before a planned restart. After reconnecting, clients should ask for the messages they missed rather than assume nothing was lost. The cost is that a few clients wait longer to reconnect.

A user's phone loses signal for 30 seconds in a chat. How do you make sure they do not miss messages?

A push connection delivers only while it is open, so the messages must also be stored. Give each message an increasing ID or sequence number per conversation and save it in a database. The client remembers the last ID it received, and on reconnect asks for everything after it. Server-Sent Events have this built in through the Last-Event-ID header. The trade-off is that you now need durable storage and an API to read history, in addition to the live push path.

How would you scale a service that must hold millions of open WebSocket connections?

Each open connection costs memory and a file descriptor, so a single server holds a limited number, and the way to scale is to add servers. Balance new connections by the number each server already holds (least connections), not round robin. Keep the servers stateless apart from their connections, and use pub/sub to carry messages between them. Tune operating system limits, use long proxy timeouts and keep-alive messages, and spread reconnect storms with jitter. The trade-off is operating many stateful, long-lived connections against the lower connection cost of polling.

Real-time Communication | TheLearningLoop

Q: Two users in the same chat room are connected to different servers. How does a message from one reach the other?

The server that receives the message publishes it to a pub/sub channel for the room, using something like Redis pub/sub or a message broker. Every server that has a user in that room is subscribed to the channel, so each one receives the message and pushes it down its own connections. The sending server does not need to know which server holds which user. The trade-off is an extra moving part, and plain pub/sub does not store messages, so a disconnected user needs the message saved elsewhere to catch up later.

Why you need real-time communication, and when to use it

Normal web traffic works like a question and an answer: the client asks, the server answers, and the connection is done. That is a poor fit when something happens on the server and the client needs to know now: a chat message, a score, a driver's position, a collaborator's edit. The server cannot speak first, so the client has to find a way to hear about changes.

The techniques in this lesson give you two things interviewers always ask about:

Low latency: how soon a client learns about new data, from "within a few seconds" to "instantly".
Scalability: how many clients you can keep informed at once, which depends on the cost of each request or each open connection.

Use real-time communication when:

Users expect to see updates without refreshing: chat, notifications, live scores, dashboards.
Several users work on the same thing at once: shared documents, multiplayer games, whiteboards.
Data moves and must be tracked live: delivery tracking, ride-hailing, stock prices.

Skip it, or use something simpler, when:

Updates are rare and a delay of a minute is fine: ordinary requests or a refresh button are enough.
The data is the same for everyone and changes slowly: short polling against a cached endpoint is cheap and simple.
The user is not on the page, such as a closed phone app: use push notifications through the phone's operating system.

Try it

Three clients, A, B and C, want to hear about messages sent at ticks 2, 5, 6, 9 and 13. Choose how they listen. Then inject a failure: the network drops at tick 8 for three ticks. The table shows when each client received each message.

Hearing about new messages in real time

Step 1 of 21 · Press Space to play, arrow keys to step

Failure:

No messages yet. Press Play.

Each client asks the server "anything new?" every 4 ticks, at different moments. A message waits until the next ask. Three clients (A, B, C) listen for messages. One tick is one unit of time.

requests sent:: 0
empty answers:: 0
average delay:: -
worst delay:: -
messages lost:: 0

1 tick = one unit of time. A delay of +3 means the message reached the client 3 ticks after it was sent. Requests count every ask a client makes, including empty ones.

Speed

Things to try:

Short polling: each client asks every 4 ticks. Over the run that is 15 requests, 4 of them empty, and messages arrive an average of about 1.5 ticks late, up to 3 ticks.
Long polling: every message arrives instantly, and no request is empty. In this short run it makes about as many requests (18), because each answer needs a new request, but each one carries news. Over a long quiet period it would make far fewer than frequent polling.
WebSocket or Server-Sent Events: just 3 connections and no delay, plus a few small keep-alive messages.
Then switch on the network failure: short polling delays messages by up to 6 ticks but loses none. Long polling recovers 2 ticks late.
Push, with "Just listen for new ones": the message sent at tick 9 is lost for all three clients. Switch to "Ask for missed messages": it arrives at tick 11, 2 ticks late, and nothing is lost.

Real-time strategies

Strategy	How it works	Good when	Weakness
Short polling	The client asks "anything new?" on a timer	Updates are rare or a delay is fine. Easy to cache and to debug	Wasted empty requests. Delay up to the interval
Long polling	The client asks and the server holds the request open until there is news	You need near-instant updates but only plain HTTP is available	A new request after every answer. Servers hold many open requests
Server-Sent Events (SSE)	One long HTTP response that the server keeps writing events into	One-way updates: feeds, scores, progress bars. Browsers reconnect automatically	Server to client only. Limits on connections per browser over HTTP/1.1
WebSockets	One two-way connection, started from an HTTP request and then upgraded	The client also sends often: chat, games, shared editing	Stateful connections. Needs proxy support and care when scaling
Push notifications	The phone's operating system delivers a short message to the device	The app is closed or the screen is off	Not guaranteed or instant. Goes through Apple's or Google's service

Pull vs push

This is the main fork in the road.

Pull (short and long polling): the client starts every exchange. It works with plain HTTP, through every proxy and cache, and the server holds no state between requests. The cost is delay and wasted requests.
Push (SSE, WebSockets): the server can speak first over a connection that stays open. Delivery is immediate and efficient, but every open connection costs the server memory and ties a client to one server, so scaling, deploying and failing over all become harder.

A good rule: start with the simplest option that meets the delay you need. Polling first, then SSE, and WebSockets only when the client also sends often.

How push connections work

A push connection is long-lived and stateful, so most of the design is about what happens around it.

Connect. A WebSocket starts as a normal HTTP request that asks to upgrade. The server agrees with a 101 Switching Protocols response, and the same connection now carries messages in both directions. SSE simply keeps an HTTP response open.
Authenticate once, when the connection is made, before any data flows.
Keep alive. The two sides send a tiny message every so often (a heartbeat). It shows the connection is still alive, and it stops proxies and load balancers from closing a connection that looks idle.
Fan out across servers. A connection lives on one server. If user A is on server 1 and user B is on server 2, a message from A must reach server 2. Servers share messages through a pub/sub layer: each server subscribes to the channels its connected users care about, and the server that receives a message publishes it to the channel. Every subscribed server then pushes it down its own connections.
Reconnect. Connections drop all the time: networks change, servers restart, proxies time out. The client reconnects with exponential backoff (wait 1 second, then 2, then 4, up to a limit) plus a little random jitter, so thousands of clients do not all retry at the same instant.
Catch up. A plain connection replays nothing, so messages sent while a client was away are gone unless you stored them. Give each message an increasing ID, store it, and have the client ask for everything after the last ID it saw. SSE has this built in: the browser sends a Last-Event-ID header when it reconnects.

Real-time communication vs the alternatives

Concern	Push connection	Short polling	Long polling	Push notification to a phone
Delay	Immediate	Up to the polling interval	Near immediate	Usually quick, not guaranteed
Wasted requests	None	Many, mostly empty	Some	None
Server cost per client	A held connection	A request now and then	A held request	Almost none
Works when the app is closed	No	No	No	Yes
Works through any proxy and cache	Needs configuration	Yes	Mostly	Not relevant
Easy to scale and deploy	Harder	Easiest	Medium	Easy (a service does it)
Best for	Chat, games, live data	Rare or cacheable updates	Real time on plain HTTP	Alerts to a closed app

Push connections win when delay matters and the user is watching. Polling wins when updates are rare, because it is simpler and costs nothing between requests.

Avoiding a new single point of failure

A real-time setup adds several things that can fail, and one failure can hit everyone at once:

The connection servers. If a server dies, all its clients drop and reconnect together. Spread connections over several servers with a least-connections balancer, and make clients reconnect with backoff and jitter so they do not overload the rest.
The pub/sub layer. If it is down, messages stop crossing servers. Run it with replicas or as a cluster, and consider storing messages durably so clients can catch up when it recovers.
The load balancer. It must support long timeouts and the upgrade headers, and it should be redundant too.
The message store. The catch-up path depends on it. A message that was only pushed and never stored is lost for anyone who was offline.

The trade-off you accept: holding stateful connections and an extra pub/sub layer, in exchange for instant delivery. Deploy by draining a server gradually (stop accepting new connections, let the old ones move) rather than restarting it in one go.

Common mistakes

Leaving proxy timeouts at their defaults. NGINX closes an idle upstream connection after 60 seconds unless told otherwise, and many load balancers do something similar. Raise the timeouts and send heartbeats.
Forgetting the upgrade headers. A WebSocket through a proxy that does not pass Upgrade and Connection fails with a plain HTTP error.
Reconnecting immediately. After a restart, every client retries at once and overloads the servers. Use exponential backoff with jitter.
Assuming nothing is lost. A push connection replays nothing. Store messages with IDs and let clients ask for what they missed.
Balancing connections by round robin. Connections last hours and vary in activity, so round robin piles them unevenly. Use least connections.
Sending big messages or no limits. One client sending huge or endless messages can use up a server. Limit message size and rate.
Using WebSockets where SSE or polling would do. A two-way channel nobody uses is extra complexity to operate.

How to talk about real-time communication in an interview

Ask what the client needs: how fresh must the data be, does the client send too, how many users at once? Then pick the simplest option that fits and say why.
Name the transport and its cost: "SSE for one-way updates, WebSockets for chat, short polling if a few seconds of delay is fine".
Explain fan-out: connections live on one server, so a pub/sub layer carries messages between servers.
Mention failure unprompted: reconnect with backoff and jitter, heartbeats, and catch-up by message ID so nothing is lost.
State the trade-off: instant delivery against the cost of holding many stateful connections and the harder deployments that come with them.

Where you meet real-time communication in real systems

Whenever a page updates without you touching it, something on this list is at work. Three examples are drawn out below, the first with configuration you can run, then more.

Example 1: A chat service behind a reverse proxy

A chat app runs on several servers, and each user keeps a WebSocket open to one of them. A reverse proxy or load balancer sits in front. Left at its defaults it would cut quiet connections after a minute and refuse the upgrade request, so the main job is configuring it for long-lived connections: pass the upgrade headers, allow long timeouts and spread connections by how many each server holds. Open the example below to see the picture and the configuration for three tools.

Config: a reverse proxy for WebSocket and event-stream trafficSetup 1 config fileCode no code changes

Each browser opens one long-lived connection to the proxy. The proxy passes the WebSocket upgrade through and chooses the chat server with the fewest connections. The servers share messages with each other through a pub/sub layer.

# Complete nginx.conf: proxies WebSocket and Server-Sent Events traffic to two chat servers.
# Addresses are placeholders (10.0.0.x is a private range); replace them with your servers.
events {}

http {
    # A WebSocket starts as a normal HTTP request with "Upgrade: websocket".
    # Forward "upgrade" when the client asked for it, and "close" for ordinary requests.
    map $http_upgrade $connection_upgrade {
        default upgrade;
        ''      close;
    }

    upstream chat_servers {
        least_conn;                       # spread long-lived connections by how many each server holds
        server 10.0.0.11:8080;
        server 10.0.0.12:8080;
    }

    server {
        listen 80;

        # WebSocket endpoint.
        location /ws/ {
            proxy_pass http://chat_servers;
            proxy_http_version 1.1;                          # the upgrade needs HTTP/1.1
            proxy_set_header Upgrade    $http_upgrade;       # pass the upgrade request through
            proxy_set_header Connection $connection_upgrade;
            proxy_set_header Host       $host;
            proxy_read_timeout 3600s;                        # default is 60s, which would cut quiet connections
            proxy_send_timeout 3600s;
        }

        # Server-Sent Events endpoint (one-way stream from server to browser).
        location /events/ {
            proxy_pass http://chat_servers;
            proxy_http_version 1.1;
            proxy_set_header Connection "";                  # keep the upstream connection open
            proxy_buffering off;                             # send each event at once instead of collecting them
            proxy_read_timeout 3600s;
        }
    }
}

Each configuration was checked with the tool itself in a throwaway container: nginx -t on the NGINX stable image, haproxy -c on the HAProxy lts image, and envoy --mode validate on Envoy 1.31. A deliberately broken NGINX file and a broken HAProxy file were rejected, so the checks are real. The 10.0.0.x addresses, ports and paths are placeholders. These examples use plain HTTP to stay short, so in production put TLS in front (the wss:// form of WebSockets) before any user data or credentials flow through.

Example 2: Live scores over Server-Sent Events

A sports site shows live scores to many fans. Scores only flow from the server to the browser, so the browser opens one Server-Sent Events stream and the server writes an event each time something happens. If the connection drops, the browser reconnects by itself and tells the server the last event it saw, so it can catch up.

Each browser keeps one HTTP response open. The score service writes an event into every stream when a goal is scored. After a drop, the browser reconnects and sends the ID of the last event it received.

Example 3: A message crossing servers

Two friends are in the same chat room but their connections landed on different servers. The first friend's message reaches server 1, which publishes it to the room's channel. Server 2 is subscribed to that channel, receives the message and pushes it down its own connection to the second friend. Neither server needs to know where the other user is.

Alice is connected to server 1 and Bob to server 2. Alice's message goes to server 1, which publishes it. Server 2 receives it from the pub/sub layer and pushes it to Bob. The highlighted path is the message's journey.

More real-world scenarios

4. Collaborative editing

In a shared document, each edit goes to a server over a WebSocket and is sent on to everyone else viewing it, so cursors and changes appear within a fraction of a second.

5. Multiplayer games

A game needs to send player actions often and receive the world's state many times a second. WebSockets, or lower-level protocols for fast-paced games, give the two-way low-delay channel it needs.

6. Delivery and ride tracking

A driver's phone reports its position every few seconds, and the customer's app shows the car moving. The server pushes each new position to the customer, so the map updates without the app polling.

7. Stock prices and live dashboards

A trading screen or monitoring dashboard shows numbers that change constantly. Servers push each change, and clients may drop updates they are too slow to show, because only the latest value matters.

8. Notifications

When a post is liked or an order ships, the web app updates a badge instantly over a live connection while the page is open. When the app is closed, the same event goes out as a push notification through the phone's operating system.

9. Progress of long-running jobs

A video upload or a report that takes minutes to build shows a progress bar. The server streams progress events, with Server-Sent Events being enough because the data flows one way.

Tools and managed services

Kind	Name	Notes
Open source	NGINX, HAProxy, Envoy	Reverse proxies and load balancers that can pass WebSocket upgrades and hold long connections.
Open source	Redis Pub/Sub	A simple publish and subscribe system often used to fan messages out across servers. It does not store messages.
Open source	Socket.IO	A JavaScript library for real-time messaging that can fall back to long polling when WebSockets are not available.
Framework	ASP.NET SignalR	A .NET library for real-time communication over WebSockets, SSE or long polling.
Managed	AWS API Gateway WebSocket APIs	Hosts WebSocket connections for you and calls your backend on each message.
Managed	Azure SignalR Service	Runs the connections for SignalR apps so your servers do not hold them.
Managed	Firebase Realtime Database and Cloud Firestore	Clients listen to data and receive changes as they happen.
Managed	Pusher and Ably	Hosted services for real-time messaging between servers and clients.

The takeaway

Whenever a design must show fresh data without a refresh, the first question is how often it changes and who is listening. The first box you draw is usually polling for slow data, a stream for one-way updates and a WebSocket for chat, with a pub/sub layer once there is more than one server.

Real-time Communication

Key takeaways