Design a News Feed

Who Asks This Question?

The news feed question is a staple at social media companies and any platform with user-generated content. Based on interview reports, it's frequently asked at:

• Facebook/Meta — Obviously their core product; they want to see if you understand timeline generation at scale
• Twitter — Multiple reports of this exact question, focusing on real-time delivery and celebrity accounts
• Instagram — Asked with emphasis on media handling and mobile optimization
• LinkedIn — Professional feed variant with career-related content prioritization
• TikTok — Modern take focusing on algorithmic feed ranking and video processing
• Pinterest — Board-based feeds with image-heavy content discovery
• Snapchat — Stories-based feed with ephemeral content considerations
• YouTube — Subscription feeds mixed with recommendations

What the Interviewer Is Really Testing

Most candidates focus on the database schema: "Users follow users, posts go in tables." That's maybe 15% of what matters. Here's the actual scoring breakdown at most companies:

Step 1: Clarify Requirements

Questions You Must Ask

These questions fundamentally change your architecture:

"Is this more like Facebook (friend connections) or Twitter (asymmetric follows)?" This determines your fan-out strategy. Symmetric friend networks are easier to scale because relationships are mutual. Asymmetric follows create the celebrity problem.

"How many followers can a user have? What's the maximum?" This is the most important question. If the answer is "unlimited" or "millions," you need a hybrid fan-out approach. If it's "5,000 max," you can use simpler fan-out on write.

"Is the feed chronological or algorithmic?" Chronological feeds are easier to implement but less engaging. Algorithmic feeds require ranking signals, ML models, and much more complex cache invalidation.

"What types of content: text, images, videos?" Video changes everything. You need CDNs, transcoding pipelines, and different storage strategies. Images need resizing and optimization. Text is the easiest.

"How fresh does the feed need to be?" Real-time (like Twitter during live events) requires push notifications and WebSocket connections. Eventually consistent (like Facebook) allows more aggressive caching.

Requirements You Should State

After asking questions, be explicit about what you're building:

Functional:

• Users can follow other users
• Users can create posts (text, images, optional)
• Generate a personalized feed showing posts from followed users
• Support basic interactions: likes, comments, shares

Non-functional:

• 100 million daily active users
• Each user follows 200 people on average (but some celebrities have millions of followers)
• 1 million posts created per day
• Feed loads in under 200ms
• Eventually consistent feed updates (posts appear within 5 minutes)

Step 2: High-Level Architecture

The Fan-Out Decision

This is the central system design choice. There are three approaches, and your choice depends entirely on the scale and user behavior:

Fan-out on Write (Push Model): When someone posts, immediately push that post to all their followers' feeds. Each user has a pre-computed feed that's ready to display.

Fan-out on Read (Pull Model): When someone requests their feed, pull posts from all the people they follow and merge them in real-time.

Hybrid (The production answer): Use fan-out on write for normal users, fan-out on read for celebrities.

Let's walk through each:

Fan-Out on Write (Push Model)

User A publishes a post
    |
    v
[Fan-out Service] reads A's followers list
    |
    +---> Push to User B's timeline cache
    +---> Push to User C's timeline cache  
    +---> Push to 1000 other followers' caches

Advantages:

• Fast reads: feed is pre-computed
• Good for users who read more than write (most social apps)

Disadvantages:

• Expensive writes: posting to 1M followers requires 1M cache updates
• Celebrity problem: when a celebrity posts, it can overwhelm the fan-out service
• Storage cost: you're storing the same popular post in millions of timelines

When to use: Small to medium scale, or when users have limited follow counts

Fan-Out on Read (Pull Model)

User B requests feed
    |
    v
[Timeline Service] reads B's following list  
    |
    +---> Query posts from User A (last 100)
    +---> Query posts from User C (last 100)
    +---> Merge and rank posts
    +---> Return top 20 to user

Advantages:

• No celebrity problem: posting is O(1) regardless of follower count
• Storage efficient: each post stored once
• Fresh content: always shows latest posts

Disadvantages:

• Slow reads: must query hundreds of users on every feed request
• Expensive for users who follow many people
• Hot key problem: celebrity timelines get hammered with reads

When to use: When you have celebrities with millions of followers, or write-heavy workloads

Hybrid Model (Production Approach)

The best production systems use both:

Fan-out Rules:
  - Users with < 1M followers: use fan-out on write
  - Users with > 1M followers: use fan-out on read
  - Mix both in the final feed assembly

Flow:

1. Normal user posts → fan out to all their followers immediately
2. Celebrity posts → store in their timeline, don't fan out
3. When generating a feed → fetch pre-computed timeline + pull from celebrity accounts user follows

This gives you fast reads for normal content and avoids the celebrity bottleneck.

Core Architecture Components

[Client App]
    |
    v
[Load Balancer]
    |
    +---> [Timeline Service] ← generates feeds
    +---> [Post Service] ← creates/stores posts
    +---> [Fan-out Service] ← distributes posts to followers
    +---> [Media Service] ← handles images/videos
    |
    v
[Timeline Cache] (Redis) ← pre-computed feeds
[Post Database] (MySQL/Postgres) ← post storage
[Media Storage] (S3) ← images, videos  
[Social Graph] (Redis/Neo4j) ← following relationships

Step 3: Deep Dive — Fan-Out Strategy Implementation

Fan-Out on Write: The Implementation

When a user publishes a post, you need to deliver it to potentially millions of followers without blocking the publisher.

class FanOutService:
    def fan_out_post(self, post_id: str, author_id: str):
        # Get author's followers (this could be millions)
        followers = self.get_followers(author_id, batch_size=1000)
        
        # Fan out in batches to avoid overwhelming any single worker
        for follower_batch in self.batch(followers, 1000):
            self.queue.publish(FanOutBatch(post_id, follower_batch))
    
    def process_fan_out_batch(self, batch: FanOutBatch):
        for follower_id in batch.follower_ids:
            # Add post to follower's timeline cache
            self.timeline_cache.add_to_feed(follower_id, batch.post_id)

Key optimization: Use message queues (SQS, Kafka) to make fan-out asynchronous. The post publisher shouldn't wait for fan-out to complete.

def publish_post(user_id: str, content: str) -> str:
    # Store the post (synchronous)
    post_id = self.post_db.create_post(user_id, content)
    
    # Start fan-out (asynchronous)
    self.fan_out_queue.publish(FanOutTask(post_id, user_id))
    
    # Return immediately to user
    return post_id

Timeline Cache Structure

Each user's timeline is cached as a sorted list of post IDs, ordered by timestamp:

# User 123's timeline cache
TIMELINE:123 = [
    "post:789:1640995200",  # post_id:timestamp 
    "post:456:1640995100", 
    "post:123:1640995000"
]

# Keep only the most recent 1000 posts per user
LTRIM TIMELINE:123 0 999

Why store post IDs instead of full posts? Because the same post appears in millions of timelines. Store the full post once, reference it everywhere.

Fan-Out on Read: The Implementation

class TimelineService:
    def get_feed(self, user_id: str, limit: int = 20) -> List[Post]:
        # Get users this person follows
        following = self.social_graph.get_following(user_id)
        
        # Pull recent posts from each person they follow
        post_candidates = []
        for followed_user in following:
            recent_posts = self.post_db.get_user_posts(
                followed_user, limit=50
            )
            post_candidates.extend(recent_posts)
        
        # Sort by timestamp and take top N
        sorted_posts = sorted(post_candidates, 
                            key=lambda p: p.timestamp, 
                            reverse=True)
        
        return sorted_posts[:limit]

Performance problem: If a user follows 500 people, this queries 500 different database partitions. That's way too slow.

Solution: Batch the queries and cache aggressively:

def get_feed_optimized(self, user_id: str, limit: int = 20) -> List[Post]:
    following = self.social_graph.get_following(user_id)
    
    # Check cache first
    cached_posts = self.get_cached_recent_posts(following)
    
    # Batch query for any cache misses
    missing_users = [u for u in following if u not in cached_posts]
    if missing_users:
        fresh_posts = self.post_db.batch_get_user_posts(missing_users)
        self.cache_recent_posts(fresh_posts)
        cached_posts.update(fresh_posts)
    
    # Merge and rank
    all_posts = []
    for user, posts in cached_posts.items():
        all_posts.extend(posts)
    
    return self.rank_posts(all_posts)[:limit]

The Celebrity Problem

When a celebrity with 10M followers posts, fan-out on write becomes a disaster:

• 10M cache updates
• Takes minutes to complete
• Overwhelms the fan-out service
• Other users' posts get delayed

Solution: Hybrid approach with celebrity detection:

class HybridFanOutService:
    CELEBRITY_THRESHOLD = 1_000_000
    
    def handle_new_post(self, post_id: str, author_id: str):
        follower_count = self.social_graph.get_follower_count(author_id)
        
        if follower_count < self.CELEBRITY_THRESHOLD:
            # Normal user: fan out to all followers
            self.fan_out_to_followers(post_id, author_id)
        else:
            # Celebrity: just mark as available for pull
            self.mark_celebrity_post(post_id, author_id)
    
    def get_hybrid_feed(self, user_id: str) -> List[Post]:
        # Get pre-computed feed from normal users
        precomputed = self.timeline_cache.get_timeline(user_id)
        
        # Get fresh posts from celebrities user follows
        celebrities = self.get_celebrity_follows(user_id)
        celebrity_posts = self.get_celebrity_posts(celebrities)
        
        # Merge and rank
        all_posts = precomputed + celebrity_posts
        return self.rank_posts(all_posts)

Step 4: Deep Dive — Ranking and Personalization

Beyond Chronological: Why Ranking Matters

Chronological feeds are simple but perform poorly:

• Users miss important posts if they don't check frequently
• Popular content gets buried by time
• No personalization based on interests

Modern feeds use ranking algorithms to surface the most relevant content.

Ranking Signals

class FeedRanker:
    def calculate_post_score(self, post: Post, user: User) -> float:
        score = 0.0
        
        # Recency: newer posts get higher scores
        hours_old = (now() - post.created_at).total_seconds() / 3600
        recency_score = max(0, 1 - (hours_old / 24))  # Decay over 24h
        score += recency_score * 0.3
        
        # Engagement: posts with more likes/comments rank higher  
        engagement_score = math.log(1 + post.like_count + post.comment_count)
        score += engagement_score * 0.3
        
        # Relationship strength: closer friends rank higher
        relationship_score = self.get_relationship_strength(user.id, post.author_id)
        score += relationship_score * 0.2
        
        # Content type preferences
        if post.has_media and user.prefers_media:
            score += 0.1
        
        # Topic relevance (if you have ML models)
        if hasattr(post, 'topics'):
            topic_score = self.calculate_topic_relevance(post.topics, user.interests)
            score += topic_score * 0.1
        
        return score

Relationship strength can be computed from:

• How often the user likes/comments on this author's posts
• How recently they've interacted
• If they're in each other's "close friends"
• Mutual friends/followers

Machine Learning Integration

Production systems use ML models for ranking, but you don't need to design the ML infrastructure in a system design interview. Just show you know it exists:

class MLFeedRanker:
    def __init__(self):
        # In production, this would be a served ML model
        self.ranking_model = load_model("feed_ranking_v3")
    
    def rank_posts(self, posts: List[Post], user: User) -> List[Post]:
        # Extract features for each post
        features = []
        for post in posts:
            feature_vector = self.extract_features(post, user)
            features.append(feature_vector)
        
        # Get relevance scores from ML model
        scores = self.ranking_model.predict(features)
        
        # Sort by predicted relevance
        post_scores = list(zip(posts, scores))
        post_scores.sort(key=lambda x: x[1], reverse=True)
        
        return [post for post, score in post_scores]

Features you might extract:

• Post metadata: type, length, media presence, creation time
• Author metadata: follower count, verification status, posting frequency
• User-author interaction history: likes, comments, message exchanges
• Content analysis: topics, sentiment, engagement patterns

Pagination Strategy

Don't use offset-based pagination for feeds — it's inefficient and inconsistent when new posts are added.

Bad approach:

SELECT * FROM posts WHERE user_id IN (...) 
ORDER BY created_at DESC 
LIMIT 20 OFFSET 40;  -- Page 3

Problem: If new posts are added while the user is paginating, they'll see duplicates or miss posts.

Good approach: Cursor-based pagination:

class FeedPagination:
    def get_feed_page(self, user_id: str, cursor: str = None, limit: int = 20):
        if cursor:
            # Continue from where we left off
            cursor_time = self.decode_cursor(cursor)
            posts = self.get_posts_before(user_id, cursor_time, limit)
        else:
            # First page: start from most recent
            posts = self.get_most_recent_posts(user_id, limit)
        
        # Generate cursor for next page
        if posts:
            next_cursor = self.encode_cursor(posts[-1].created_at)
            return FeedPage(posts, next_cursor)
        
        return FeedPage(posts, None)
    
    def encode_cursor(self, timestamp: datetime) -> str:
        # Base64 encode the timestamp
        return base64.b64encode(timestamp.isoformat().encode()).decode()
    
    def decode_cursor(self, cursor: str) -> datetime:
        timestamp_str = base64.b64decode(cursor).decode()
        return datetime.fromisoformat(timestamp_str)

API response:

{
  "posts": [...],
  "next_cursor": "MjAyNC0wMS0xNVQxNDozMDowMFo=",
  "has_more": true
}

Media Handling

Text posts are easy. Images and videos change the system significantly:

Image Processing Pipeline:

User uploads image
    |
    v
[API Gateway] validates file type/size
    |
    v
[Upload Service] stores original in S3
    |
    v
[Image Processing Queue] triggers async processing
    |
    v
[Image Processor] generates multiple sizes:
    - Thumbnail: 150x150
    - Medium: 600x400  
    - Large: 1200x800
    |
    v
[CDN] serves optimized images globally

Video is much more complex:

• Transcoding to multiple formats (MP4, WebM)
• Multiple resolutions (720p, 1080p, 4K)
• Adaptive bitrate streaming (HLS/DASH)
• Thumbnail generation
• Content moderation

For the interview, just mention: "Videos would require a transcoding pipeline with services like AWS Elemental or FFmpeg, plus CDN distribution for global low-latency access."

Common Mistakes

Mistake 1: Not Considering the Celebrity Problem

Designing fan-out on write without thinking about what happens when someone with millions of followers posts. In reality, this breaks the system. Strong candidates immediately ask about follower distribution and mention the hybrid approach.

Mistake 2: Ignoring Content Ranking

Assuming feeds are chronological in 2024. Modern social platforms are algorithmic. Even if you keep the ranking simple, show you understand that relevance matters more than recency.

Mistake 3: Storing Full Posts in Timeline Caches

Storing complete post data in every user's timeline cache is incredibly wasteful. Store post IDs in timelines, fetch full post data separately. This allows you to update post metadata (like count) without invalidating timeline caches.

Mistake 4: Single Database for Everything

Putting posts, social graph, and timeline data in one database creates bottlenecks. Different data has different access patterns:

• Posts: write-heavy, mostly append-only
• Social graph: read-heavy, highly connected
• Timeline caches: read/write-heavy, can tolerate some data loss

Use different storage systems optimized for each use case.

Mistake 5: No Caching Strategy

Feed generation touches multiple services and databases. Without aggressive caching, every feed request becomes expensive. Cache at multiple layers:

• CDN for media files
• Redis for timeline caches
• Application-level cache for user profiles and post metadata

Interviewer Follow-Up Questions

"How would you handle content moderation?" Build it into the publishing pipeline, not as an afterthought. Before fan-out, run posts through automated content detection (spam, hate speech, nudity). Flag suspicious content for human review. For feeds, have a "content filter service" that can remove posts from timelines after publication.

"What if someone deletes a post?" With fan-out on write, you need a "fan-out deletion" service that removes the post from all followers' timeline caches. With fan-out on read, deletion is automatic since you're always fetching fresh post data.

"How do you handle real-time notifications when someone likes your post?" Separate system from the feed. Use WebSocket connections or push notifications. The notification service listens to like events and delivers them to the post author in real-time.

"What about trending topics or hashtags?" Track hashtag usage in a separate service. Use a time-windowed counter (similar to rate limiting) to identify trending topics. Popular hashtags can be promoted in the feed ranking algorithm or shown in a separate "trending" section.

"How would you A/B test different ranking algorithms?" Version your ranking models and randomly assign users to different cohorts. Track engagement metrics (time spent, likes, comments) per cohort to measure which ranking performs better. Use feature flags to gradually roll out new algorithms.

Summary: Your 35-Minute Interview Plan