TheLearningLoopWhy you need a load balancer
One server can only handle so many requests. When traffic grows, you add more servers, but clients still need one address to talk to. A load balancer sits in front of the servers, accepts every request, and forwards each one to a healthy server.
It gives you two things interviewers always ask about:
- Scalability: add servers behind it to handle more traffic (horizontal scaling).
- Availability: if a server dies, health checks notice and traffic goes to the others.
Try it
Send requests and switch between algorithms. Then take a server down and watch the load balancer route around it. Servers have different capacities, and some requests are slower than others, so the algorithm you pick changes how evenly the load spreads.
Load balancer
Step 1 of 55 · Press Space to play, arrow keys to stepRequest #1 (slow, 14 units) → server A: next in turn (A → B → C).
- tick:
- 0
- completed:
- 0
- failed:
- 0
- avg response (ticks):
- -
- busiest server queue:
- 1
1 tick = one step of the simulation.
Things to try:
- Run round robin with mixed request sizes. Notice how one server can pile up slow requests while others sit idle.
- Switch to least connections and run the same traffic. The queues stay much more even.
- Mark server B as down mid-run. Its traffic moves to A and C, and no requests are sent to B.
Load balancing algorithms
| Algorithm | How it picks a server | Good when | Weakness |
|---|---|---|---|
| Round robin | Next server in the list, in turn | Servers and requests are similar | Ignores current load |
| Weighted round robin | Like round robin, but bigger servers get more turns | Servers have different capacity | Still ignores current load |
| Least connections | Server with the fewest active requests | Request durations vary a lot | Needs live connection counts |
| IP hash | Hash of the client IP picks the server | You need a client to keep hitting the same server | Uneven if a few clients send most traffic |
| Power of two choices (pick two at random) | Pick two at random, send to the less busy one | Very large fleets | Slightly less even than least connections |
Layer 4 vs layer 7
- A layer 4 (L4) load balancer routes by IP address and port only. It is very fast but cannot see the HTTP request.
- An L7 load balancer reads the HTTP request, so it can route
/apito one pool and/imagesto another, handle HTTPS encryption so the servers don't have to (called TLS termination), and add headers.
How load balancer health checks work
The load balancer calls a small status URL, such as GET /healthz, on each server every few seconds. After a few failures in a row, the server is taken out of rotation. After a few successes, it comes back. This is what makes the system self-healing.
Avoiding a new single point of failure
A single load balancer is itself a single point of failure. Real setups run two or more load balancers, for example an active-passive pair (one handles traffic, and the other takes over its IP address if it fails, called a floating IP), or DNS pointing at several of them.
How to talk about load balancers in an interview
- Draw it early: clients, then the load balancer, then several stateless app servers.
- Say which algorithm you would use and why. Least connections is a safe default when request times vary.
- Mention health checks, keeping servers stateless (store login sessions in a shared store such as Redis, not in one server's memory, so any server can handle any request), and running the load balancer redundantly.