CoursesSystem design from first principles to design review
Puzzle 18 · 19 of 19
If one driver gives up, whose read should stop?
Give two callers different deadlines and trace what survives the first timeout.
All puzzles in this course
The problem
The circuit breaker currently allows calls. Ada and Ben ask the same worker for the west map. Request coalescing gives them one pending storage read. Ada can wait 400 milliseconds. Ben can wait six seconds. Storage has not replied when Ada's time runs out. When Ada gets a timeout, is Ben still waiting? What must happen before his six-second deadline for him to receive the map?
See it
Scroll sideways to see the full diagram.
Ada and Ben have joined the same lookup. The three lines show each caller's wait and the underlying storage work.
Step 1 of 4Ada's deadline expires before storage replies. Her line ends at the cross. Ben still has time left. Predict what he can receive.
Step 2 of 4Storage completes once. Ben's wait ends with a successful reading. Ada's earlier timeout stays a timeout.
Step 3 of 4One result served the remaining waiter. If the callers had used different service instances, both could still receive these replies even though storage did twice the work.
Step 4 of 4
The building block
Separate three lifetimes: Ada's wait, Ben's wait and the storage operation. A single storage read can supply both callers. Each caller waits for whichever happens first: its own deadline or the shared result. When Ada's deadline wins, the service can send her a timeout. Ben still holds a reference to the shared work.
Think of two people waiting for the same printed train timetable. Ada leaves to catch a bus. Ben is still at the counter, so the clerk keeps printing. If expensive work should stop when everyone leaves, the clerk needs to know how many people remain. That is a separate policy.
Run the two callers
The browser starts both waits and releases the deadline and storage result in a controlled order. It models the deadline as an explicit event so the result does not depend on how fast your phone is. Before pressing Run scenario, predict Ben's reply if storage succeeds before his deadline and how many storage reads will have started. Then run both callers on one instance. Then change only the routing so each caller reaches a different instance. Predict which column changes. The reads column is cumulative, including the fresh lookup in the last row.
if (!pending.has(lot)) {
reads++;
const work = storageReady.then(() => '200 ' + lot);
pending.set(lot, work.finally(() => pending.delete(lot)));
}
return Promise.race([pending.get(lot), deadline]);
Read the complete runnable code
async function compareWaiters(split) {
let releaseStorage, expireCaller;
let reads = 0;
const storageReady = new Promise(resolve => { releaseStorage = resolve; });
const shortDeadline = new Promise(resolve => { expireCaller = resolve; });
const noDeadlineInThisTrace = new Promise(() => {});
function makeInstance() {
const pending = new Map();
return function read(lot, deadline) {
if (!pending.has(lot)) {
reads++;
const work = storageReady.then(() => '200 ' + lot);
pending.set(lot, work.finally(() => pending.delete(lot)));
}
return Promise.race([pending.get(lot), deadline]);
};
}
const first = makeInstance();
const second = split ? makeInstance() : first;
const short = first('west', shortDeadline);
const long = second('west', noDeadlineInThisTrace);
expireCaller('504 timeout');
const shortReply = await short;
const rows = [{ event: 'deadline', short: shortReply, long: 'waiting', reads }];
releaseStorage();
const longReply = await long;
rows.push({ event: 'storage replies', short: shortReply, long: longReply, reads });
const later = await second('west', noDeadlineInThisTrace);
rows.push({ event: 'later lookup: ' + later, short: 'finished', long: 'finished', reads });
return rows;
}The code first looks up the pending work for the lot. The last line creates a separate wait for each caller. Winning that race does not cancel the other promise. Each instance has its own list of pending operations. A later call after cleanup starts a new read because this fixture has no completed-answer cache.
Try it
Does this outcome also occur across real HTTP connections? Save the local experiment as waiters.py and run the command below with Python 3. The script starts temporary HTTP servers on your own computer. Each service instance has its own pending table. They run in the same Python process and share an executor for the storage calls.
python3 waiters.py
The script holds the storage response until both driver requests have entered their handlers and the short wait has ended. The following lines are the decision in the handler. The timeout belongs to the caller waiting on the future, the object holding the eventual answer. The storage worker continues running. The status is 504 Gateway Timeout because the map service is acting as a gateway: it answers for a storage read it is waiting on.
try:
return result.result(timeout=0.4 if budget == "short" else 6)
except concurrent.futures.TimeoutError: # the same class as TimeoutError from Python 3.11
return 504, "caller deadline"
Compare your result
On one instance, the server returns 504 to Ada and 200 to Ben after one overlapping storage read. Across two instances the driver replies stay the same, but storage receives two reads. Both runs also verify a successful fresh lookup after cleanup. The difference comes from the location of the pending table.
What if both drivers leave? This implementation still lets an already-started storage read finish. To justify canceling it, count remaining waiters under the same ownership rule as registration and check that storage actually stopped the work. A client timeout alone cannot establish that the underlying work stopped.
Check yourself
Sources
- Python · Future.result — “this method will wait up to timeout seconds”
- MDN · Promise.race
What you now know
One caller can stop waiting while a shared operation still serves another caller.
Answer every question above correctly to unlock the next puzzle.
In the final project, decide which caller may stop waiting, which operation continues and what evidence the next attempt must consult.
Apply what you learned
Continue with worked examples and practice cases. When you reach the final project, make your own decisions before comparing your evidence with the worked review.
Continue to Guarantees under load and partial failure → Start the final project →
Something wrong or unclear on this page? Tell me: a puzzle that didn't click is a bug in the puzzle.