1 The model cannot see words

## Watch your words break apart

Before a model does anything, your text is chopped into tokens — pieces from a fixed list of about 50,000. Common words get one each. Rare ones get shattered.

Type anything — a gene, a drug, a sentence

2 Meaning becomes distance

## Does the model know these mean the same?

To compare text, a model turns each phrase into a point in space and measures the distance. 1.00 means identical. Pick a pair and see whether it agrees with you.

Choose a pair

3 How the next word is chosen

## Turn the dial

The model does not produce a word. It produces a score for every word it knows. A setting called temperature decides how adventurous the final pick is.

Unfinished sentence
Temperature — 0.7

Left: it always says the same thing — repeatable, and dull. Right: unlikely words become live options — varied, and more often wrong. Nothing about the model changed. Only the dial.

4 The bug you probably have

## It only read half the note

Before blaming a model for poor search results, check something simpler: how much of each document did it actually read?

Our synthetic discharge summaries

The popular general-purpose model accepts 256 tokens. Our notes average over 500. It read the first half of each one and discarded the rest — with no error and no warning. The half it threw away holds the hospital course and the discharge medications.