LLMs for mobile engineers, from tokens to your app · check

Know Text retrieval and evidence already?

One question for each of the 2 puzzles in this module. A right answer marks that puzzle as passed, so you can jump ahead. A wrong one tells you which puzzle to start with. Nothing is sent anywhere: it's saved in this browser.

Puzzle 8: Find evidence before asking the model

The query and article express the same idea with different words. What should you measure?

Right. Hold the corpus fixed and compare the rankings.

Not quite. Neither method in this run generates answers. Learn it in puzzle 8 →

Not quite. The problem is selecting evidence from the collection. Learn it in puzzle 8 →

Puzzle 9: Which step failed the answer?

The expected article is second of three results. Does this question count as a hit at 3?

Not quite. First place is the cutoff for hit at 1, not hit at 3. Learn it in puzzle 9 →

Right. Pick the cutoff before measuring.

Not quite. This retrieval metric is computed independently of the answer. Learn it in puzzle 9 →

Back to the overview →