Why you need multi-region and disaster recovery, and when to use them
A data centre can lose power, a cloud region can have an outage, a bad deployment can delete data, and a flood can destroy a building. Disaster recovery (DR) is the plan to bring the service back after such an event. A multi-region design runs the service in more than one geographic region so one region's failure does not stop it.
It gives you two things interviewers always ask about:
- Availability: users keep getting service, or get it back quickly, when a whole region fails.
- Durability and latency: data survives the loss of a region, and users near each region get faster responses.
Two numbers define the target:
- RTO (recovery time objective): how long you can be down.
- RPO (recovery point objective): how much recent data you can afford to lose, measured in time.
RTO is counted from the moment the service breaks, so it includes the time to notice the failure and decide to act, not only the time to switch over. The minutes and hours in this lesson are illustrative round numbers, not measurements of any real system.
Use multi-region when:
- The cost of a regional outage is higher than the cost of the extra regions.
- You have users around the world and need low latency near each.
- Rules require data to stay within a country or region.
Keep it simple (one region plus backups) when:
- The business can tolerate hours of downtime.
- The service is small, and the extra complexity would cause more outages than it prevents.
Try it
A region fails, or a bug deletes data. Compare three setups.
Multi-region and Disaster Recovery
Step 1 of 4 · Press Space to play, arrow keys to stepInitial state. Region A serves everyone and copies its writes to a standby Region B, a little behind.
- tick:
- 0
- downtime (minutes):
- 0
- data lost (minutes of writes):
- 0
1 tick = one step of the simulation.
Things to try:
- Run One region + nightly backup with Region A goes down. Recovery takes hours and loses hours of data.
- Switch to Active-passive, then Active-active, and compare downtime and data lost.
- Choose A bug deletes data with each setup. Notice that replication copies the mistake.
What to watch: the two counters are the RTO ("downtime") and the RPO ("data lost"). The simulation assumes the newest nightly backup is 10 hours old, a restore takes 4 hours, the standby is 1 minute behind and a failover takes 10 minutes. These are illustrative assumptions. Active-active shows 0 minutes of downtime as a simplification: in a real system, clients with cached DNS answers and requests in flight would still see a short disruption. A bad delete shows the same loss for every setup because the simulation has no point-in-time recovery, which would cut that loss to a few minutes.
Disaster recovery patterns
| Pattern | How it works | RTO and RPO | Cost and weakness |
|---|---|---|---|
| Backup and restore | Copy data elsewhere regularly. Rebuild after a disaster | Hours, hours | Cheapest, slowest |
| Pilot light | A minimal copy of the system keeps running, scaled up on demand | Under an hour, minutes | Low cost, needs a tested scale-up |
| Warm standby (active-passive) | A reduced copy runs and receives replicated data | Minutes, seconds to minutes | Pays for idle capacity, failover must be practised |
| Active-active | Several regions serve users at the same time | Near zero | Highest cost, conflicts to resolve, harder to build |
Active-passive vs active-active
- Active-passive: one region serves all traffic. A standby region receives copies of the data and takes over on failure. Failover is a deliberate step (promote the standby, move traffic), so there is a short outage, and writes not yet copied are lost.
- Active-active: both regions serve users, so a failure is a traffic shift and users barely notice. The hard part is data: writes in two regions can conflict, so you need partitioning by user, conflict resolution, or a database that handles multi-region writes.
- Capacity: a failover only helps if the surviving region can carry the load. With two active regions, each must be able to take all the traffic, so each runs at most about half full. With three regions, each can run at about two thirds full, because the two survivors then carry 1.5 times a normal share (1.5 x 2/3 = 1). A warm standby or pilot light must be able to scale up fast, which means having quota and capacity reserved in advance, since many customers may ask for the same capacity during a regional disaster. (Utilisation figures are simple arithmetic, not a vendor guarantee.)
Synchronous or asynchronous copying
How the data reaches the second region decides the RPO.
- Asynchronous: the primary confirms a write to the user first and copies it afterwards. Writes are fast, but anything not yet copied when the region fails is lost, so the RPO equals the replication lag.
- Synchronous: the primary waits for the other region to confirm before it confirms to the user. The RPO is close to zero, but every write pays a cross-region round trip (tens to a couple of hundred milliseconds, depending on distance), and if the other region is unreachable, writes may have to stall or the system must fall back to asynchronous copying.
- Consensus databases (see consensus and coordination) wait for a majority of replicas, so, with replicas in at least three regions, they survive one region failing without losing confirmed writes, at the same write-latency cost.
This is the consistency-versus-latency trade-off from CAP theorem and consistency, applied across regions. The simulation above models asynchronous copying.
How a failover works
- Detect: health checks from outside the region decide the region is unhealthy. Avoid flapping by requiring several failures.
- Decide: automatic for clear failures, a person for ambiguous ones, since a wrong failover can cause split brain. Split brain means two regions both believe they are the primary and both accept writes, so their data diverges and must be reconciled later.
- Promote the data: the standby database becomes the primary. Replication lag decides how much recent data is missing.
- Move traffic: change DNS records, anycast (one address announced from several regions, so the network sends users to a nearby healthy one) or a global load balancer. Keep DNS time-to-live short, and remember some clients cache.
- Fail back: after the primary region recovers, resynchronise data and return traffic deliberately, not by accident.
- Practise: run regular game days (planned drills in which you fail a region on purpose). An untested plan usually fails when needed.
Backups still matter
Replication copies every change, including mistakes and corruption. A deleted table is deleted in all regions within seconds. Only backups and point-in-time recovery protect against that. Keep backups in another account or region, protect them from deletion, and test restores.
Multi-region design vs the alternatives
| Concern | Multi-region active-active | Warm standby | One region + backups |
|---|---|---|---|
| Survives a region failure | Yes, with little or no downtime | Yes, with a short outage | Yes, after hours |
| Data lost on failure | Small, with conflicts | Small | Hours |
| Protects against bad deletes | Only with backups | Only with backups | With backups |
| Cost | Highest | Medium | Lowest |
| Complexity | Highest | Medium | Lowest |
| Best for | Global, critical services | Important services | Everything else |
Choose the cheapest pattern that meets the RTO and RPO the business actually needs.
Avoiding a new single point of failure
- Failover tooling: the thing that moves traffic must not live only in the region that fails. Run it elsewhere.
- DNS and global load balancing: use a provider that is separate from your region.
- Secrets and configuration: replicate them too, or the standby cannot start.
- Dependencies: a shared service in one region (a payment provider, an identity service) can tie all regions to one failure.
- Data residency: if law keeps data in a country, the standby region must be inside the allowed area too, or the recovery plan itself breaks the rule.
You accept higher cost and complexity for resilience, and a risk of mistakes during failover.
Common mistakes
- Replication mistaken for backup. Mistakes replicate.
- Untested failover. The plan fails on the day.
- Forgetting dependencies. The service is up, but sign-in or payments live in the failed region.
- Long DNS caching. Traffic keeps going to the dead region.
- Active-active without a conflict plan. Two regions edit the same record.
- No fail-back plan. Returning is as risky as leaving.
How to talk about multi-region and DR in an interview
- Ask for the RTO and RPO, and what a regional outage costs.
- Pick a pattern to match: backup and restore, warm standby, or active-active, and explain the data part (replication lag and conflicts).
- Mention backups and point-in-time recovery for human error, health checks, DNS or global load balancing, and regular failover drills.
- Name the trade-off: lower RTO and RPO cost more money and complexity.
- Say whether the copy is synchronous or asynchronous (RPO against write latency), and check that the surviving region has the capacity to take the full load.