Mobile system design, from a packet to an app · Start here · 1 of 11

What actually travels

Your app calls an API and gets JSON back, as if the two were joined by a pipe. Underneath, there's no pipe: only small chunks of data that can get lost. Here's the stack of layers that turns them into a request and a response.

The problem

In your code, a request is one call, and a user object comes back. But the phone and the server are joined by Wi-Fi, a cell tower, cables and a dozen routers, machines that pass data along toward its address, any of which can drop data. How does a request survive that trip, and what does it cost?

See it

A request is wrapped by TLS and carried by TCP as numbered packets; IP routes each packet through routers and one gets lost; the receiver's acknowledgement shows the gap and TCP sends the lost packet again; the server gets the request in order app server TCPTLSHTTP request IP: routers, best effort 1 2 lost 2not acknowledged: sent again
  1. Your request is HTTP, encrypted by TLS (it scrambles the data), carried by TCP: layers inside layers.

    Step 1 of 4
  2. TCP cuts it into numbered packets. IP passes each one from router to router: best effort, no promises.

    Step 2 of 4
  3. Packet 2 is lost on the way. Predict: who notices, and what happens?

    Step 3 of 4
  4. TCP: packet 2 was never acknowledged, so it's sent again. The server gets everything, in order.

    Step 4 of 4

The building block

Nothing travels as one piece. Data is cut into packets: small chunks, each up to about a kilobyte and a half, each with an address on it.

IP, the Internet Protocol, is the postal service: it addresses each packet and passes it from router to router toward the server. It does its best, nothing more. A packet can be lost, delayed, or overtaken by the one sent after it.

TCP, the Transmission Control Protocol, builds something reliable on top. The sender numbers every packet; the receiver sends back acknowledgements, "got up to number 12"; anything not acknowledged in time is sent again; the receiver puts packets back in order. Your code sees one clean stream of bytes. To keep that bookkeeping, both sides must first agree to talk: a TCP connection, set up with a short exchange before any data flows.

On top of TCP, two more layers:

LayerIts job
HTTPthe request and the response (puzzle 2)
TLSencrypts the stream, so routers on the way can't read or change it (puzzle 4)
TCPreliable, ordered delivery, over a connection
IPaddresses and routes each packet, best effort

Every puzzle that follows is about the cost of this stack: how long packets take to cross (puzzle 3), what setting up a connection costs (puzzle 5), and what the phone's radio pays to send them (puzzle 6).

The analogy

Sending a book by post, one page per envelope. The post office (IP) delivers most envelopes, in any order, and loses a few. Your friend (TCP) numbers the pages, writes back "got pages 1 to 40", and you resend the missing ones. Before the first page, you agree by letter that you're sending a book at all: the connection.

Check yourself

A packet of your request is lost between two routers. Who notices and fixes it?

Right. Numbering, acknowledging, resending: TCP.

Not quite. Your code sees a clean stream; TCP handles losses underneath.

Not quite. IP only does its best; recovering lost packets is TCP's job.

Why must a TCP connection be set up before the first request?

Not quite. IP delivers every packet anyway; the setup is TCP's bookkeeping.

Right. Reliability needs shared bookkeeping.

Not quite. TLS runs on top of TCP, but the setup is TCP's own need.

What's the difference between IP and TCP?

Right. Best effort vs reliable.

Not quite. That's the two swapped. Re-read that part ↑

Sources

  • High Performance Browser Networking: Building blocks of TCP ("The IP, or Internet Protocol, is what provides the host-to-host routing and addressing, and TCP, or Transmission Control Protocol, is what provides the abstraction of a reliable network running over an unreliable channel"; TCP hides "retransmission of lost data, in-order delivery, congestion control and avoidance, data integrity")

What you now know

Data crosses the internet in packets. IP addresses each packet and routes it, but can lose, delay or reorder it. TCP turns those packets into a reliable, ordered stream: it numbers them, waits for acknowledgements and resends what's lost, and needs a connection set up first. TLS encrypts that stream, and HTTP is the language of requests and responses on top.