Mobile system design, from a packet to an app · Puzzle 3 · 4 of 11

Round trips, not bandwidth

Your profile screen loads in a blink on office Wi-Fi and crawls on the train. The data is 60 KB. Here's what the user is actually waiting for, and why a faster network barely helps.

The problem

Your profile screen makes four requests, one after another: the user, their settings, their avatar URL, then the avatar itself. Only the avatar really needs something from the one before; the code just happens to wait for each. Together they carry 60 KB. On office Wi-Fi it's instant; on a train it takes over a second. The train's network can carry megabytes per second. So where does the second go?

See it

A phone sends four requests to a server one after another, each waiting for the last, so the wait is four round trips; doubling bandwidth barely changes it; sending independent requests together takes about one round trip phone server one round trip: 250 ms 4 KB user → settings → avatar URL → avatar4 trips in a row: 1 second twice the bandwidth: still about 1 second three together, then the avatar2 trips: 500 ms
  1. A round trip: the request goes to the server and the answer comes back. Each way takes the network's latency; this network's round trip is 250 ms.

    Step 1 of 4
  2. Your screen makes four requests, each waiting for the one before. Four trips in a row: a full second, for a few kilobytes.

    Step 2 of 4
  3. Now the network's bandwidth doubles. The trips take just as long. Predict: what does make it faster?

    Step 3 of 4
  4. Fewer trips in a row: send the three that don't depend on each other together, then the avatar. Or ask for the whole screen in one request.

    Step 4 of 4

The building block

A network link has two numbers. Latency is how long one packet takes to get from sender to receiver: distance, plus the queues and radios on the way. Bandwidth is how much data the link carries per second.

A request needs a round trip: the request travels to the server, the server answers, the answer travels back. For a few kilobytes, the data itself takes almost no time to send. Almost all the wait is the trip.

So for small requests, the time is roughly:

Time
One request1 round trip
Four requests, each waiting for the one before4 round trips
The three independent ones together, then the avatar2 round trips

At 30 ms per round trip on Wi-Fi, four in a row is 120 ms. At 250 ms on a busy mobile network, it's a full second, and doubling the bandwidth doesn't change it, because you weren't waiting for bandwidth. Sending the user, settings and avatar URL requests together, then the avatar, takes two round trips: 500 ms. (In Kotlin, start them with async and wait with awaitAll; over one connection this needs a newer HTTP version, puzzle 5.)

That's why the first question of any mobile system design is: how many round trips does this screen need, and can they happen at the same time, or not at all? One endpoint that returns the whole screen, or requests that don't depend on each other, beats any amount of compression.

The analogy

Asking a friend in another city four questions by post, one letter at a time, waiting for each answer before you write the next. A bigger envelope doesn't help. Putting all four questions in one letter does.

Check yourself

Your screen's four requests each wait for the one before, on a 250 ms mobile network. Bandwidth doubles. Roughly how long does loading take now?

Not quite. Sharing bandwidth doesn't shorten a trip.

Not quite. For a few kilobytes the data takes almost no time; the trips do.

Right. Small requests wait for round trips.

Which change makes that screen faster on every network?

Right. Fewer trips in a row, faster everywhere.

Not quite. More bandwidth doesn't remove round trips; fewer trips do.

Not quite. Smaller data saves little when the wait is the trip, not the data.

What's the difference between latency and bandwidth?

Right. Time of the trip vs size of the pipe.

Not quite. That's the two swapped. Re-read that part ↑

Sources

What you now know

Latency is how long a packet takes to arrive; bandwidth is how much data fits through per second. Small requests are bound by latency, so the wait is the number of round trips times the round-trip time. Requests that wait for each other add their round trips together; cut the round trips, and the screen gets faster on every network.