Why video streaming is hard, and what it teaches
Video is large, viewers have very different networks, and everyone wants it to start at once and never pause. A video streaming service stores a video once, converts it into many versions, spreads copies close to viewers, and lets each player pick the best version for its current connection.
It gives you two things interviewers always ask about:
- Smooth playback (latency and quality): quick start, no buffering spinner, and the best picture the network allows.
- Scalability and cost: serving millions of viewers must not need millions of requests to the origin, because bandwidth is the main cost.
Requirements to confirm first:
- Functional: upload a video, watch it on many devices, search and resume, optionally live streaming.
- Non-functional: start in a second or two, few stalls, huge read traffic, and durable storage of the originals.
Terms used below: the origin is your own storage or servers that hold the real files, as opposed to cached copies. A bit rate is how much data one second of video needs, measured in Mbps (megabits per second). A higher bit rate gives a better picture but needs a faster network.
Skip the design, or simplify it, when:
- You have a few short clips: serve the file from object storage with a CDN in front.
- Audience is small and on good networks: one quality is enough.
Try it
A viewer watches 14 seconds of video. Change the player's behaviour and the network.
Video Streaming
Step 1 of 16 · Press Space to play, arrow keys to stepInitial state. The player has 2 seconds of video buffered. Each tick it plays one second and downloads whatever the network allows.
- tick:
- 0
- stalls:
- 0
- seconds stalled:
- 0
- average quality (Mbps):
- -
1 tick = one step of the simulation.
Things to try:
- Run Fixed quality with the bandwidth drops network. From tick 6 the buffer falls every second, and the video stalls once, for one second, on the last tick. It stalls late because the buffer had built up before the drop. An earlier or deeper drop would stall sooner.
- Switch to Adaptive bitrate. The player lowers the quality and keeps playing.
- Try Bandwidth flaps with both players and compare stalls with average quality. Neither player stalls here, because a buffer of a few seconds absorbs a short dip. The adaptive player still lowers its quality (about 3.3 Mbps on average, against 5.0 for the fixed player), so on brief dips adaptive buys nothing and costs picture quality. Its benefit shows on longer drops.
What to watch: the Buffer box is the player's safety margin, and it is what decides whether a stall happens. This simulation uses a rate-based rule (pick the highest quality that fits within 80 percent of the measured speed). Real players also use buffer-based rules (move down when the buffer is low), and many combine both. The simulation is simplified to one second per tick, not whole segments.
Streaming approaches
| Approach | How it works | Good when | Weakness |
|---|---|---|---|
| Download the whole file | The player gets one big file before playing | Small clips | Slow start, wasted data if the user leaves |
| Progressive download | A single file is played while it downloads | Simple sites | Cannot change quality mid-way |
| Adaptive bitrate (HLS, DASH) | The video is cut into short segments in several qualities. The player picks per segment | Most streaming | More storage and a transcoding pipeline |
| Live streaming | An encoder sends segments continuously with a short delay, set by segment length and the player's buffer | Events and sports | Delay and sudden audience spikes |
Origin only vs CDN
- Origin only: every viewer fetches segments from your storage. It is simple, but your servers and bandwidth bill grow with the audience, and far-away viewers are slow.
- CDN (content delivery network): copies of popular segments are cached on servers near viewers. A cache hit never reaches your origin. Segments are small, static files, which suits caching very well. The price is cache invalidation and CDN cost.
How adaptive streaming works
- After upload, a transcoding pipeline converts the video into several resolutions and bit rates (a ladder), each cut into segments of a few seconds.
- A manifest file lists the versions and their segments. The player downloads it first.
- The player measures how fast segments arrive and keeps a buffer of a few seconds.
- For each next segment it picks the highest quality that fits the measured speed. When the buffer falls, it drops to a lower rung. When the network recovers, it goes up.
- Segments are plain files over HTTP, so any web server or CDN can serve them.
Streaming design vs the alternatives
| Concern | Adaptive bitrate with a CDN | Single file from storage | Streaming through your app servers |
|---|---|---|---|
| Handles different networks | Yes | No | Possible |
| Load on your servers | Low | Medium | High |
| Fast start | Yes | Slow | Medium |
| Storage used | High (many versions) | Low | Low |
| Extra moving parts | Transcoding, CDN | None | Custom servers |
| Best for | Real video products | Small sites | Almost never |
Use adaptive bitrate and a CDN for anything beyond a few clips.
High-level design
- Write path (upload and processing): the client uploads in chunks to object storage. An event goes on a queue, transcoding workers produce the ladder, the segments, a manifest and thumbnails, and then mark the video ready in the metadata database.
- Read path (playback): the player asks an API for the video's metadata and manifest address, downloads the manifest from the CDN, and then fetches segments from the CDN. The CDN fetches from the origin only on a miss.
- Metadata, search and resume: a database holds title, owner, status, manifest location and per-user resume position. Search runs on a separate index fed from that database. These are small, ordinary data and are not on the segment path.
- Live streaming: an encoder sends segments continuously and the manifest keeps changing. Delay depends on segment length and how much the player buffers, and shorter segments lower it at the cost of more requests.
Capacity estimation
These numbers are illustrative assumptions, used to show the method. Use the interviewer's numbers when they give them.
- Storage per hour of video: the ladder in this lesson is 1, 3 and 5 Mbps, which is 9 Mbps in total. 9 Mbps for 3,600 seconds is 32,400 megabits, or 4,050 MB, so about 4 GB per hour of video, before the original file. Real ladders have more rungs.
- Storage growth: if 10,000 hours are uploaded each day, that is 10,000 x 4.05 GB = 40,500 GB, so about 40 TB a day or about 14.8 PB a year.
- Files: with 2-second segments, one hour is 1,800 segments per rung, so 5,400 segment files for three rungs. A 2-second segment at 3 Mbps is 6 megabits, so 0.75 MB.
- Playback egress: 1 million viewers at an average of 3 Mbps is 3,000,000 Mbps, which is 3 Tbps. No single origin can serve that, so the CDN must.
- Origin load: with a 95 percent CDN hit rate, the origin sees 5 percent, which is 150 Gbps. At a 99 percent hit rate it is 30 Gbps. Each percentage point of hit rate matters.
- Request rate: 1 million viewers each fetching one 2-second segment every 2 seconds is 500,000 segment requests a second. At 95 percent hits, 25,000 a second reach the origin.
Avoiding a new single point of failure
- Upload and transcoding: use a queue with retries so a failed job is retried. Videos are not available until the pipeline finishes.
- Origin storage: replicate across locations. The CDN can keep serving popular videos even if the origin is briefly down.
- CDN: use more than one provider or region for important events.
- Metadata database: replicate it. The video file and the information about it live apart.
You accept extra storage for many versions, and a delay between upload and availability.
Bottlenecks and how to relieve them
- Transcoding: it is the heaviest compute in the system and bursts after uploads. Run workers from a queue and scale them with queue depth. Spend effort on popular videos first, and consider producing rarely watched rungs only when first requested.
- CDN egress and cost: bandwidth is the main bill. A higher hit rate, a mid-tier cache between edge and origin, and a sensible ladder (not too many rungs) all reduce it.
- Cache-miss storm: when a new or evicted segment is requested by many viewers at once, all the misses can reach the origin together. Request collapsing (shown in the NGINX example as
proxy_cache_lock) and pre-warming prevent it. - Hot titles: one popular video concentrates load on the same few segments. They stay in cache, so the risk is at the first moment of a release, not afterwards.
- Live delay: a shorter segment lowers delay but raises the request rate and the chance of stalls, so choose it from the delay target.
Common mistakes
- Serving large files from application servers. Use object storage and a CDN.
- Transcoding inside the upload request. Do it in the background from a queue.
- One quality only. Viewers on slow networks stall.
- Very long segments. Quality changes and startup become slow. Very short segments have their own cost: more requests, larger manifests and less efficient compression, so a few seconds is the usual compromise.
- Ignoring cache keys. Different URLs for the same segment defeat the cache.
- No plan for a viral spike. Pre-warm caches and limit origin requests.
How to talk about video streaming in an interview
- Split the system into upload and processing (write path) and playback (read path).
- Explain transcoding to a ladder of qualities, segments with a manifest, and adaptive bitrate in the player.
- Put object storage at the origin and a CDN in front, and mention a queue for transcoding.
- Name the trade-off: more storage and processing in exchange for smooth playback on any network.
- Give rough numbers: storage per hour of video, egress for the audience, and origin load at a stated hit rate.
Related topics: caching, blob and object storage, message queues and async and real-time communication.