CoursesLLMs for mobile engineers, from tokens to your app

Practice chapter 2 of 2

Tools, reusable teaching routines and production behavior

Connect a read-only lookup under an explicit permission

A tool definition describes an operation the model may request. The host validates the arguments, checks authorization, executes or declines, then returns a result tied to the request. The model never gets to grant itself permission. Start with a test tool reading only a small approved lesson catalog.

Worked example

The user enables lookup for lesson L4. The model requests read_lesson(L4); the host returns its title and evidence. A later request for L7 is outside the session’s allowed lesson IDs and is declined. Log the operation, decision, elapsed time and result category without including private message text.

Try it

Where should a test assert that no lookup ran after permission was withdrawn?

Compare your answer

At the host boundary. Verify zero executions and a logged denial even if a model still proposes the call. A polished assistant reply is insufficient evidence of correct permission enforcement.

Source · Claude documentation · Tool use

Compare a skill with a tool on the same questions

A skill packages reusable instructions and optional supporting resources. A tool is a callable capability exposed by the host. A teaching skill might specify “ask for a prediction, inspect evidence, explain the discrepancy”; it can use the same authorized lookup tool as an ordinary prompt.

Worked example

Run ten saved learner questions through two configurations. Give both the same model, allowed source material and tool permissions. Vary only the teaching instructions. Compare evidence IDs, support for factual claims and whether the second example tests transfer.

Try it

One configuration is more fluent but invents two citations. Does the teaching routine earn promotion?

Compare your answer

Score source support separately from style, inspect each missing or invented citation and retest on held-out questions. Reusable instructions cannot create evidence or expand permissions.

Source · Agent Skills · Format specification

Set a latency budget and a useful fallback

Measure time to first visible text and time to a complete checked response. Define a service objective in terms of the learner’s task. A fast stream can still produce a slow or unusable final answer. Bound retrieval, model generation and validation separately so failure in one stage leaves time for a fallback.

Worked example

For a five-second budget, allocate 0.5 s for network and authentication, 0.7 for retrieval, 3.2 for generation and 0.6 for final checks. These are design targets. If evidence lookup is unavailable, display the previously reviewed lesson text with its version instead of presenting an unsupported generated explanation.

Try it

Two routes meet the first-token objective. One completes 98 of 100 evidence checks and the other 91. Which measurements determine release?

Compare your answer

Compare checked completion rate, factual support, latency distribution and cost on the same cases. Define a threshold before evaluating; a failed evidence check should trigger the bounded fallback rather than a confident answer.

Source · Google SRE · Implementing SLOs