LLMs for mobile engineers, from tokens to your app · Puzzle 2 · 3 of 8

Your first call

Puzzle 1 said the model works in tokens. Now one real call, so everything after it has something concrete to stand on: what your app sends, what comes back, and the three numbers that matter in the response.

The problem

You know the model continues text and counts in tokens. But what does the request look like, in code you could write with OkHttp? And what comes back besides the answer?

See it

The app sends a POST with model, max_tokens and a messages list with one user turn; the response holds the text, a stop_reason and usage with input and output tokens; a cut-off answer says max_tokens your app provider POSTmodel, max_tokensmessages: [user] textstop_reasonusage: in 312, out 9 cut off: stop_reason = max_tokens you pay per token; output costs more
  1. One HTTP POST: the model's name, max_tokens, and messages, a list of turns; here, one user turn.

    Step 1 of 4
  2. Back comes JSON: the text, a stop_reason, and usage, the input and output tokens of this call.

    Step 2 of 4
  3. If the answer hits your limit, it stops mid-sentence and says so. Predict: which tokens cost more?

    Step 3 of 4
  4. Output tokens: they're priced higher, and each takes a step to generate.

    Step 4 of 4

The building block

A call to Anthropic's Messages API is one HTTP POST with a JSON body (shown slightly simplified):

{
  "model": "MODEL_NAME",
  "max_tokens": 300,
  "messages": [
    { "role": "user", "content": "Summarise this review: …" }
  ]
}
  • model: which model to use, a name from the provider's docs.
  • max_tokens: the most tokens the answer may have.
  • messages: the conversation as a list of turns, each marked user (your side) or assistant (the model's earlier answers). Puzzle 4 shows why that list matters.

The response, also JSON:

{
  "content": [ { "type": "text", "text": "Short battery life, great screen." } ],
  "stop_reason": "end_turn",
  "usage": { "input_tokens": 312, "output_tokens": 9 }
}

Three things to read every time:

  • the text in content;
  • stop_reason: end_turn means the model finished; max_tokens means it was cut off at your limit, mid-sentence;
  • usage: the input and output tokens of this call. That's the bill: providers charge per token, and output tokens cost several times more than input ones. Output is also what takes time, one token per step (puzzle 1).

The analogy

A telegram service that charges per word, with a higher rate for the reply. You state the maximum reply length up front, and if the reply hits it, it stops, mid-sentence.

Check yourself

The response's stop_reason is max_tokens. What happened?

Not quite. That's a different error; this is about the answer's length.

Not quite. That's end_turn; max_tokens means it hit your limit.

Right. Cut off at your limit.

A call used 312 input and 9 output tokens. Which part costs more per token?

Not quite. Output tokens are priced higher.

Not quite. Input is cheaper per token; output costs several times more.

Right. Output costs more.

What's the difference between a user turn and an assistant turn?

Right. Your side vs the model's.

Not quite. That's the two swapped. Re-read that part ↑

Sources

  • Anthropic: Messages API examples (a response contains content, stop_reason such as "end_turn" or "max_tokens", and "usage": { "input_tokens": …, "output_tokens": … }; requests send model, max_tokens and messages with roles)
  • Anthropic: Pricing (input and output tokens are priced separately, output higher)

What you now know

A call to a model is one HTTP POST: a model name, max_tokens (the most it may write), and messages, a list of turns each marked user or assistant. The response holds the continuation, a stop_reason (end_turn when it finished, max_tokens when it was cut off), and usage: the input and output tokens you pay for. Output tokens cost more than input ones, and each one takes time to generate.