CoursesLLMs for mobile engineers, from tokens to your app
Practice chapter 1 of 2
What a call costs and how it fails
Budget an answer and inspect why it stopped
An output token limit caps how much the model may generate. It is a ceiling rather than a desired length. A transport success can still carry a result cut off at that limit. Inspect the provider’s stop reason before treating text as complete or handing it to a downstream parser.
Worked example
A lesson requests three worked cases but receives only two and half of a third. The stop reason reports max_tokens. Display the completed portion as partial. Shorten the requested unit of work or issue an explicitly budgeted continuation with enough context to avoid duplicating the first two cases.
Try it
A response contains valid-looking JSON followed by the maximum-token stop reason. Which checks precede saving it?
Compare your answer
Check completion, parse it, validate the expected fields and enforce business constraints. Retry within the remaining time and cost budget if the contract requires a complete answer.
Source · Claude documentation · Stop reasons
Price input and generated output separately
A provider can charge different prices per input and output token. Prompt processing and sequential generation have different execution patterns; the actual prices are provider policy, not a universal ratio derivable from token count. Estimate with the selected model’s published rates and verify against recorded usage.
Worked example
Use fictional rates of 2 units per million input tokens and 8 per million output tokens. A request with 2,000 input and 500 output tokens costs 0.004 + 0.004 = 0.008 units. Output is one fifth of the tokens and half of the example bill.
Try it
You can remove 500 input tokens or 200 output tokens without reducing quality. Which saves more at these rates?
Compare your answer
The input reduction saves 0.001 units per call. The output reduction saves 0.0016. Multiply by measured traffic and include cache reads and writes when applicable.
Source · Claude documentation · Pricing
Retry inside a deadline
Rate limits constrain access over time. A submitted call may exceed request or token throughput even when account spending remains available. Treat a rate-limit response as a scheduling signal. Respect the service’s retry delay, add bounded backoff where appropriate and track the overall deadline across attempts.
Worked example
The phone has a three-second deadline. The first attempt takes 0.8 s, then the service asks for a five-second delay. Retrying inside this interaction cannot satisfy its deadline. Return a usable fallback and permit an explicit later attempt.
Try it
What happens when each of five devices independently retries a failed call three times immediately?
Compare your answer
They can create up to fifteen extra attempts together. Apply limits at the proxy, avoid synchronized retries, and stop abandoned work when the user leaves. Retrying a text request can also incur another charge.
Source · Claude documentation · Rate limits
Cache a reusable prompt prefix
Prompt caching reuses eligible input computation for a common prefix. Put stable policy and shared evidence before frequently changing messages. Cache lifetime, minimum eligible prefix size and prices depend on the service and model. A cache hit still requires generating an answer to the current request.
Worked example
Two students ask different questions about the same long manual. Reuse the unchanged manual prefix and put each student’s question after it. If a timestamp is inserted at the start on every call, much of the later prefix no longer matches. Inspect cache write and read usage rather than inferring hits from response speed.
Try it
In fictional units a prefix costs 5 to write and 1 per cache hit, compared with 4 per ordinary read. What is the total for three calls before expiry?
Compare your answer
One write plus two hits costs 7, versus 12 without caching. Include cache misses and changing traffic when forecasting production cost.
Validate the meaning after validating the shape
Structured output constrains the response to a supported schema. Define fields, required properties and allowed alternatives. Parse only after a completed response and separately verify values that depend on your application’s state. A well-formed document can cite a nonexistent lesson.
Worked example
The schema requires a text field lesson_id and a list of evidence IDs. The model returns lesson_id L4 with evidence X9. Parsing succeeds. The course contains L4 but the retrieval result contained only X2 and X3, so the application rejects the unsupported citation.
Try it
A grading result has score 3 and maximum 2. Both fields are integers. Where should the inconsistency be caught?
Compare your answer
Validate the relation between values in application code and reject the result before displaying a grade. Keep the schema, input evidence and validation reason with the test fixture.