LLMs for mobile engineers, from tokens to your app · check

Know A model in your app already?

One question for each of the 8 puzzles in this module. A right answer marks that puzzle as passed, so you can jump ahead. A wrong one tells you which puzzle to start with. Nothing is sent anywhere: it's saved in this browser.

Puzzle 0: What a language model is

What does a language model do with the text you send it?

Right. It continues text.

Not quite. Your text is input to continue, not code to run. Learn it in puzzle 0 →

Not quite. It doesn't look things up; it continues text based on what it learned. Learn it in puzzle 0 →

Puzzle 1: One token at a time

Why does a 300-token answer take longer than a 30-token one?

Not quite. It doesn't reread for sending; it generates one token per step. Learn it in puzzle 1 →

Right. One token per step.

Not quite. A few kilobytes cost little; the time is in generating each token. Learn it in puzzle 1 →

Puzzle 2: Your first call

The response's stop_reason is max_tokens. What happened?

Not quite. That's a different error; this is about the answer's length. Learn it in puzzle 2 →

Not quite. That's end_turn; max_tokens means it hit your limit. Learn it in puzzle 2 →

Right. Cut off at your limit.

Puzzle 3: Why answers stream

Streaming is turned on. What gets faster?

Not quite. The bytes are about the same; only when they're shown changes. Learn it in puzzle 3 →

Not quite. The model writes at the same speed; you just see it sooner. Learn it in puzzle 3 →

Right. Sooner, not faster.

Puzzle 4: It remembers nothing

Your app sends only the latest message. Why does the bot forget the device the user just named?

Right. Stateless: send what it should know.

Not quite. It keeps nothing because the API is stateless, not by choice per message. Learn it in puzzle 4 →

Not quite. One message is tiny; the earlier one wasn't in the request. Learn it in puzzle 4 →

Puzzle 5: Same question, different answers

You set temperature to 0. Can you now assert the exact output string in a test?

Not quite. Anthropic's docs say even at 0 the results won't be fully deterministic. Learn it in puzzle 5 →

Right. Test properties, not strings.

Not quite. Prompt length doesn't make the output deterministic. Learn it in puzzle 5 →

Puzzle 6: Your key is not a secret

The key is in the APK, obfuscated by R8. Is it safe?

Not quite. R8 renames code; the key is still a value the app sends. Learn it in puzzle 6 →

Right. If the app can send it, users can read it.

Not quite. HTTPS protects the wire from others, not from the phone's owner. Learn it in puzzle 6 →

Puzzle 7: On the phone or in the cloud

Which feature suits an on-device model best?

Right. Short, frequent, private.

Not quite. A long, hard task suits a cloud model behind your backend. Learn it in puzzle 7 →

Not quite. Long documents and hard reasoning need a bigger, cloud model. Learn it in puzzle 7 →

Back to the overview →