Design YouTube / Video Streaming

Video Streaming Mechanics

A video streaming service like YouTube or Netflix splits the system into two distinct workflows:

1. Upload & Transcoding Pipeline: Accepting large video files and processing them into multiple resolutions and formats.
2. Streaming Delivery Pipeline: Delivering chunks of video to clients dynamically based on network speeds.

1. Transcoding Pipeline (DAG)

When a raw video is uploaded, it is massive (e.g. 10GB raw 4K recording). We cannot stream this raw file directly to mobile users on 4G networks. We must transcode it.
We build a **Directed Acyclic Graph (DAG)** pipeline to run processing tasks in parallel:

• Splitter: The raw video file is split into smaller chunks (GOP - Group of Pictures) to be processed in parallel.
• Encoding tasks: The chunks are compressed using different codecs (e.g. H.264, VP9) and scaled to standard formats (1080p, 720p, 480p).
• Audio split & watermark: Audio is processed separately (adding stereo/surround formats) and watermarks are overlaid onto video frames.
• Merger: Reassembles the processed video segments and audio files into unified metadata manifest files (like .m3u8).

2. Streaming Protocol & Delivery

We do not download the entire video file at once. Instead, we stream small, sequential chunks (typically 2-10 seconds long). Two popular streaming protocols are **HLS** (HTTP Live Streaming, built by Apple) and **DASH** (Dynamic Adaptive Streaming over HTTP).

Adaptive Bitrate Streaming:

Adaptive streaming ensures the player switches video quality dynamically based on the user's current internet connection.
• The transcoder generates multiple streams (e.g., 240p, 480p, 1080p) along with an index Manifest file listing the URLs for all chunks.
• The player downloads the manifest file first. It constantly measures network speed.
• If network speed drops, the player downloads the next 5-second chunk from the 360p pool. If speed increases, it requests the 1080p pool chunk.