Design a News Feed (Twitter / Instagram)

What is a News Feed?

A news feed is a constantly updating list of status updates, photos, and videos from entities a user follows. The system consists of two major pipelines:

1. Feed Publishing: When a user publishes a post, the data is written to databases and populated to all followers' feeds.
2. Feed Generation / Retrieval: When a user refreshes their homepage, the system gathers, ranks, and returns their news feed list.

The Fan-Out Models

Fan-out is the process of delivering a post to all followers. The choice of fan-out model is the most important decision in this interview:

When a user posts: the post ID is pushed 
immediately to all followers' precomputed 
feed lists (stored in a fast Redis cache).

✓ Read path is extremely fast (O(1)). 
  Just fetch the user's cached feed.
✗ Slow writes for users with millions of 
  followers (e.g. celebrities). Replicating 
  one post to 50 million feeds creates 
  massive write latency/bottlenecks.

When a user loads their homepage: the server 
queries the graph DB for their following list, 
fetches all posts for those users, and merges/
sorts them in memory.

✓ Write path is O(1) (simply append to user's 
  post history).
✓ Never duplicates post IDs in caches.
✗ Read path is slow. Aggregating and sorting 
  posts for hundreds of users on every click 
  is highly CPU/DB intensive.

The Hybrid Solution (Interview Standard)

Modern social media systems combine both approaches to handle standard users and celebrities efficiently:

• For regular users: Use **Fan-out-on-Write**. Push the post to their followers' feed caches immediately.
• For celebrities (users with > 20k–50k followers): Bypass the push mechanism. Do not write the post to their followers' caches. Instead, write it to a dedicated celebrity post stream.
• Retrieval path (Read): When a user reads their feed, fetch their pre-computed push feed list from Redis, query the celebrity posts for anyone they follow, merge the lists, and sort them. This keeps the read path extremely fast while eliminating celebrity write bottlenecks.

Cache Architecture

Caching is what makes news feeds scale. Standard caches include:

• News Feed Cache: Stores list of post IDs for each user (typically limited to top 1,000 posts to save memory). Stored in Redis.
• Content Cache: Maps post ID to actual post details (title, text, image links, author ID). This is a shared cache, preventing duplication of data in individual feed lists.
• Social Graph Cache: Stores follower and following relationships for fast checks (often Redis Set or Neo4j).