Users in Sydney wait three seconds for images from Virginia

System Design · Foundations
10 min read, about 40 min with practice

Concept
Asked often: medium
Reviewed 2026-11

See the problem

Your news site’s servers are in Virginia. A reader in Sydney opens a story with 12 photos. Each round trip across the Pacific takes about 200 ms, and the page takes 3 seconds.

You put the images behind a CDN, and the page is fast. Then you change the site logo, and Sydney still sees the old one a day later.

Define the task

What you have: one origin (your servers or object storage) in one region, and readers everywhere.
What you want: files served from a few ms away, and new versions seen at once.
The hard part: the speed comes from copies near users, and copies are what become out of date.

Pause and think

Hint 1 of 3

Once per city is enough, if the city keeps a copy.

Hint 2 of 3

Each copy has a lifetime, set by a header.

Hint 3 of 3

What if a changed file got a new name?

Understand the idea

A CDN is a chain of local shops. The factory (your origin) is far away. Each shop (an edge server) keeps popular goods in stock. The first customer of the day waits while the shop orders from the factory; everyone after gets it off the shelf.

DNS (or anycast) sends each user to a nearby edge.
On a hit, the edge answers in a few ms. On a miss, it fetches from the origin, keeps a copy and answers.
The origin’s Cache-Control header says how long a copy may be kept: max-age=86400 is one day. immutable says it will never change. private or no-store says don’t keep it at a shared cache.
A purge tells edges to drop a file now. It works, but it takes time to reach every edge and is easy to forget.

Watch an image go from one origin, to an edge, to a versioned name. Change the ocean’s round trip and the max-age below the player.

An image from Virginia, then from a Sydney edge

v1: one origin server

From Sydney
Request	Goes to	Round trip
Every user, every image	Virginia	200 ms

Step 1 of 5. Every image comes from one server in Virginia. From Sydney, each round trip takes about 200 ms. Question: 1,000 people in Sydney open the same image. How many times does it cross the Pacific?

What happens next? 1,000 people in Sydney open the same image. How many times does it cross the Pacific?

Speed Ask me first Show as text

The tests run this model of edges in front of one origin:

"""A CDN as a few edge caches near users, each in front of one origin server.

Each edge keeps a file for its max-age (from the Cache-Control header).
Round trips: about 10 ms from a user to a nearby edge, and the origin adds
its own distance on a miss. Time is passed in as `now` (seconds).
"""

from dataclasses import dataclass, field

EDGE_MS = 10  # A user to their nearby edge, there and back.


@dataclass
class Origin:
    files: dict[str, str]
    requests: int = 0


@dataclass
class Edge:
    city: str
    origin_ms: int  # This edge to the origin, there and back.
    store: dict[str, tuple[str, float]] = field(default_factory=dict)  # path -> (body, expires)

    def get(self, origin: Origin, path: str, now: float, max_age: float) -> tuple[str, int]:
        """The file and the round-trip time the user waited, in ms."""
        cached = self.store.get(path)
        if cached and now < cached[1]:
            return cached[0], EDGE_MS
        origin.requests += 1
        body = origin.files[path]
        self.store[path] = (body, now + max_age)
        return body, EDGE_MS + self.origin_ms

    def purge(self, path: str) -> None:
        self.store.pop(path, None)


def versioned(path: str, content_hash: str) -> str:
    """logo.png with hash 3f9a becomes logo.3f9a.png: a new name for every new version."""
    stem, _, ext = path.rpartition(".")
    return f"{stem}.{content_hash}.{ext}"

Climb the ladder

Each rung serves the same logo to readers in Sydney, with the origin about 200 ms away.

The idea: Every browser fetches every file from Virginia.

Browsers keep their own copies, so a returning reader is faster. A new reader pays the full cost.

Each file: about 200 ms for every file
Weak spot: the ocean, every time
Why: The same bytes cross the Pacific once per reader.

Where it breaks: distance and load. Every reader waits on the ocean, and the origin serves every byte to everyone.

The idea: Put a CDN in front, and send Cache-Control: max-age=86400 on images.

The waste in Okay: each reader fetched their own copy from across the world.

Each file: about 10 ms after the first reader in each city
Weak spot: changes take up to a day, or a purge
Why: In the tests, 1,000 readers in Sydney and some in London cost the origin 2 requests: one per city.

Where it breaks: the new logo. Edges keep the old one until max-age runs out. A purge helps, but must reach every edge, and the readers’ browsers have their own copies too.

The idea: Put a hash of the content in each file name, send max-age=31536000, immutable, and give the HTML page a short max-age.

The waste in Better: it tried to change a file under the same name, then waited for copies to notice. Most build tools add the hash for you.

Each file: about 10 ms, and updates seen at once
Weak spot: the build must rename files and rewrite links
Why: A new version is a new name, which no cache has seen, so it is always fetched fresh.

When Best isn’t best: content that changes per user, or every second (a cart, live scores), can’t be named once. Keep it out of the CDN, or cache it for a second or two.

Draw it yourself

Step 1 serves images through a CDN. Step 2 sends all traffic through it, so public pages are cached at the edge too.

Step 1: Images for users worldwide

A news site with photos. Readers are on every continent; the servers are in one region.
Serve images from near the readers, and keep them out of the database.

Solid arrow: sends requests toDashed arrow: copies data to

Watch the traffic

Send 10 requests through your board and see where each one goes. Then make a part fail and send them again.

SendReadsWritesBackground jobs

Make a part fail

How requests move

Requests start at the Client and follow your arrows. Reads try a cache first (4 in 5 are found there), then a read replica, then the database. Writes go to the database. Background jobs go on a queue, and a worker saves them. A load balancer skips a part that is down; nothing else can.

Compare

Way	Each file in Sydney	A changed file	Use it when
Origin only	about 200 ms	seen at once	internal tools, one region
CDN, one-day max-age	about 10 ms after a miss	up to a day, or a purge	files that rarely change
CDN, versioned names	about 10 ms after a miss	seen at once	scripts, styles, images
CDN, short max-age on HTML	about 10 ms within the minute	within a minute	public pages

Common mistakes and edge cases

A long max-age on the HTML page. The page is what links the new file names. If it is cached for a day, the new names are hidden for a day.
Cookies on static files. Some CDNs won’t cache a response that sets a cookie, so every request misses.
Query strings. logo.png?v=2 works on some CDNs and not others. A new file name always works.
Forgetting the origin can fail. If the origin is down, edges can serve stale copies for a while (stale-if-error), which beats an error page.
Costs. CDNs charge for bytes sent. Compress text and resize images.

Say it in the interview

Questions to ask first

Where are the users, and where is the origin?
Which content is the same for everyone, and which is per user?
How fast must a change be seen?

A 60-second walkthrough

“Readers are worldwide and the origin is in Virginia, so I’d put static files behind a CDN. Edges near users answer hits in about 10 ms; a miss goes to the origin once per edge, not once per reader. Scripts, styles and images get a hash in the file name and a max-age of a year, marked immutable. So a new version is a new name, and never stale. The HTML gets a short max-age, about a minute, for pages that are the same for everyone. Anything per user is marked private and goes straight to our servers.”

Likely follow-ups

What is an origin shield? One middle cache between the edges and the origin, so 300 edges missing at once send one request, not 300.
Can a CDN help an API? Yes, for public GET responses with a short max-age, and by ending TLS near the user.
How do you remove a file at once? Purge it. For a legal takedown, also delete it at the origin.

Red flags interviewers notice

Caching per-user pages at a shared CDN.
Updating files in place with a long max-age.
“Just purge it” as the plan for every release.

Practice

1. Cache headers (easy)

Set Cache-Control for: app.8c1d.js (a hash in the name); index.html; /api/cart; /news/today (the same for every reader, updated every few minutes).

Hint 1 of 3

A file with a hash in its name never changes under that name.

Hint 2 of 3

The HTML is what points to the new names.

Hint 3 of 3

A cart belongs to one user.

Show the answers

The idea: max-age=3600 on all four.

One setting for everything is wrong for something.

Gives: fast
Weak spot: carts leak; releases wait an hour
Why: A shared cache can give one user’s cart to another, and a new release hides behind an old index.html.

The idea: app.8c1d.js: max-age=31536000, immutable. The other three: no-store.

Safe is a good first step.

Gives: safe
Weak spot: the news page misses every time
Why: Correct, but the busiest public page always reaches your servers.

The idea: Script: max-age=31536000, immutable. index.html: max-age=60. /api/cart: private, no-store. /news/today: public, max-age=60.

Some CDNs read a separate s-maxage for shared caches, so browsers and edges can keep different times.

Gives: fast, fresh, private
Weak spot: none
Why: Names that never change cache forever; public pages cache briefly; per-user data never at a shared cache.

2. Private files (medium)

Users upload invoices (PDFs). Only the owner may download theirs, and downloads should still be fast worldwide.

Hint 1 of 3

A public CDN URL can be shared with anyone.

Hint 2 of 3

What if each link worked only for a short time?

Hint 3 of 3

Who checks that the user owns the invoice?

Show the answers

The idea: The app checks ownership, then reads the PDF from storage and sends it.

A fine start for low traffic.

Gives: safe
Weak spot: every byte goes through your servers
Why: Correct, but slow far away and heavy on the app servers.

The idea: The app checks ownership, then answers with a link that expires in 5 minutes, signed by the storage service.

S3 calls these presigned URLs.

Gives: fast download, safe
Weak spot: still from one region
Why: A leaked link stops working in minutes; the app never streams the file.

The idea: The CDN only serves requests with a valid signature and expiry, and fetches from private storage.

Keep the storage bucket private, so the CDN is the only way in.

Gives: fast worldwide, safe
Weak spot: keys to manage
Why: Edges near the user serve it, and only to whoever holds a fresh signed link.

3. Video for millions (hard)

A new episode goes live at 8 pm, and 2 million people press play within a minute.

Hint 1 of 3

Video is cut into small chunks of a few seconds each.

Hint 2 of 3

What happens at the origin when every edge misses at 8 pm?

Hint 3 of 3

Could the edges be filled before 8 pm?

Show the answers

The idea: Edges fetch chunks on the first miss.

The origin sees a stampede exactly at the busiest moment.

Gives: fine after the first minute
Weak spot: every edge misses at once at 8 pm
Why: Hundreds of edges ask the origin for the same chunk in the same second.

The idea: One middle cache between the edges and the origin, so each chunk is fetched from the origin once.

Also collapse identical requests at each edge: one fetch, many waiters.

Gives: origin load stays small
Weak spot: the first viewers still wait for the fill
Why: Edges miss into the shield, and the shield misses into the origin only once.

The idea: Push the first minutes of the episode to the edges in advance, chunked at several qualities, with a shield behind them.

Several quality levels let each player switch when the network slows (adaptive streaming).

Gives: instant start for everyone
Weak spot: storage spent on edges in advance
Why: The busiest minute is served entirely from edges that already hold the chunks.

Check yourself

Cheat sheet

Edges near users keep copies; a miss goes to the origin once per edge.
max-age sets how long a copy may be kept; immutable says it never changes; private keeps it out of shared caches.
Hash in the file name plus a year-long max-age: fast and never stale. Short max-age on HTML.
Never cache per-user responses at a shared cache.

Go deeper

Can you do these now?

If you’re not sure of any of these, scroll back to that part before moving on.

Explain how a CDN edge serves a hit and fetches a miss from the origin
Set Cache-Control for files that never change and for pages that do
Ship a new version of a file without anyone seeing the old one
Say what a CDN should not cache

Status: Not started

Users in Sydney wait three seconds for images from Virginia

See the problem

Define the task

Pause and think

Understand the idea

Climb the ladder

Serve everything from the origin

A CDN with a one-day max-age

Versioned names and a year-long max-age

Draw it yourself

Step 1: Images for users worldwide

Watch the traffic

Compare

Common mistakes and edge cases

Say it in the interview

Practice

1. Cache headers (easy)

One hour for everything

Long for the script, none for the rest

Each by what it is

2. Private files (medium)

Stream through the app

Signed URLs from object storage

Signed CDN URLs

3. Video for millions (hard)

A CDN in front of the origin

An origin shield

Pre-load the edges before 8 pm

Check yourself

Cheat sheet

Go deeper

Can you do these now?