SFB 1709 Summer School

# Basics of LLMs
and how they work

Cellular Plasticity in Myeloid Malignancies: From Mechanisms to Therapies
60 minutes slides + hands-on notebook all clinical text synthetic
How we got here

## One idea from 2017, and a great deal of electricity

Parameter counts on a logarithmic scale. Figures after GPT-3 are published estimates; the labs stopped disclosing them.

The architecture behind every model on this chart was described in a single 2017 paper. Almost nothing about the core idea has changed since.

What changed is scale, and then — once scale stopped paying for itself — training method: instruction tuning, human feedback, reasoning, and more than one kind of input.

Worth knowing today

The model we take apart in the next hour is GPT-2, from 2019. It is small, weak, and completely inspectable — which is exactly why it is the right thing to look inside.

How we got here

## “AI passes the medical licensing exam”

MedQA (USMLE-style multiple choice). Published figures: Med-PaLM 67.6% and Med-PaLM 2 86.5% (Singhal et al., 2023); GPT-4 81.4% and GPT-4 with Medprompt 90.2% (Nori et al., 2023). Passing is around 60%.

This is the headline your colleagues have read. It is also true.

What it does not mean
The exam has one right answer
Printed on the page, four distractors beside it. A ward has neither.
The questions were on the internet
Nobody can fully rule out that a model trained on the web has seen the test.
Scoring well is not the same as being safe
Nothing here measures what happens when it is confidently wrong.
In the next hour we will find three failures in these models with four lines of code each. None of them show up on this chart.
Before we start

## Three ground rules

Data

Every clinical note today is synthetic — ten discharge summaries written for this session. No PHI, no real records, no credentialed corpora. The “guideline” sources are deliberately fictional.

Numbers

Every figure in these slides came out of a real model run. Nothing is illustrative. You can reproduce all of it from the notebook.

The one idea

A language model is a machine that, given some text, produces a probability distribution over what comes next.

Everything else — the fluency, the apparent reasoning, the citations, the confident wrong answers — falls out of that one fact and the machinery that implements it.

For the next hour we take that machinery apart, in order.

Act 1 Text → Tokens

## The model has never seen a word

It sees integers. Before anything else happens, your text is chopped into pieces drawn from a fixed vocabulary of 50,257 entries.
Try:
Real GPT-2 byte-pair encoding, running in this page — 50,257 vocabulary entries and 50,000 merge rules, executed live as you type.
Act 1 Text → Tokens

## Your notation is expensive

Characters per token
—
ordinary English prose
—
cytogenetic notation

Byte-pair encoding earns a single token for sequences the training corpus saw often. Web text is full of the. It contains very little t(9;22)(q34;q11).

Token counts for the same string under four tokenizers. Lower is better.
Act 1 Text → Tokens

## What you can feed it — and how much fits

Modern models take more than text
text
images
audio
video

A blood film, a trephine, a flow plot, a karyogram — all of it can go in. But every one is converted into tokens too. An image costs hundreds to thousands of them, competing for the same limited space as your text.

The catch

“Multimodal” does not mean it reads a slide the way a haematopathologist does. It means the image joins the same queue as the words.

How much fits at once
—
of our short discharge summaries fit in a 128,000-token window. A real oncology record — years of letters, results and imaging reports — is far bigger than that.
Advertised is not usable
Windows are now enormous
Several models advertise a million tokens; a couple claim ten.
Accuracy falls long before the limit
Effective context is shorter than advertised on every model benchmarked; material in the middle is missed first.
Token counts measured from our synthetic corpus; context-window sizes are vendor figures, and the advertised-vs-effective gap is a published finding.
Act 2 Tokens → Vectors

## Meaning becomes geometry

Each token becomes a point in a few hundred dimensions, arranged so that related things land near each other. This is what powers every “find similar” feature.

Fifty haematology terms, projected to two dimensions. Switch the encoder and watch the points move.

Hovering
Hover a point to see its nearest neighbours.
Projections are PCA, Procrustes-aligned between the two models so the movement you see is real disagreement, not a rotation artefact.
Act 2 Tokens → Vectors

## Does it know AML is acute myeloid leukaemia?

general encoder biomedical encoder
The general-purpose encoder scores
—
for AML vs acute myeloid leukaemia

It does not know they are the same disease. It has mostly seen “AML” as an acronym for other things.

Build patient-similarity search on the default encoder from any vector-database tutorial, and every note that abbreviates drops out of your results — silently, with no error.

Act 2 Tokens → Vectors

## Where both encoders fail

These pairs mean opposite things
Controls — these should be close
FLT3 status decides whether the patient receives a FLT3 inhibitor. To the encoder, the two statements are the same point.
general biomedical cosine similarity, 0 to 1
Act 2 Tokens → Vectors

## The bug you will actually hit

all-MiniLM-L6-v2 — the tutorial default

The encoder accepts 256 tokens. Our notes average 527. It read the first half of every note and raised no error — no exception, no warning, just quietly worse results.

Nearest-neighbour retrieval, cluster match

Ceiling is 9/10 — our one CMML has no same-cluster neighbour. Chance is about 1.3/10.

Chunking fixes the truncation and recovers the general model. It does nothing for the biomedical one, which was barely truncating.

Clinical retrieval stays hard. Every note here mentions marrow, blasts, counts, cycles. Topical similarity is not clinical similarity.

Act 3 Attention

## Tokens read each other

A token's meaning is not fixed. Attention lets disease notice that No appeared eight tokens earlier — five of them fragments of extramedullary.
layer 0
head 0

Rows attend to columns. The lower-triangular shape is not decoration — GPT-2 is causal, so a token can only look backwards. That is what makes left-to-right generation possible.

12 layers × 12 heads
144
separate attention patterns per forward pass. Real GPT-2 weights, precomputed.
Act 3 Attention

## So why did Act 2 lose the negation?

The information was there

Some head does carry a link from the negated noun back to the negation. Attention can represent it. Attention is not the bottleneck.

Pooling threw it away

A sentence embedding averages all those token vectors into one. The 0.98 similarity was not a failure of attention — it was a failure of what we did after attention.

Practical rule: use a sentence embedding for topical search. When the clinical meaning turns on a single word, use something that reads token by token.
Act 4 Logits → Text

## The model does not output text

It outputs a distribution over all 50,257 tokens. Then something else — the sampler, which you control — picks one.
temperature 0.70
top-p 0.90
Tokens still in play
—
after temperature and top-p, out of 50,257
Entropy
—
bits — how undecided the model is
Real GPT-2 logits. The softmax is computed in this page as you move the sliders. GPT-2 is a small 2019 model with no medical training — we are watching the mechanism, not the medicine.
Act 4 Logits → Text

## Temperature is a governance setting

Most of what people describe as a model being “creative” or “unreliable” is a sampler setting, not a property of the model.

Extracting a dose from a note

temperature = 0 — you want the same answer every time, and you want to be able to say why you got it.

Drafting alternative phrasings

temperature > 0 — variation is the point.

“Why did the system output this?” is a much harder question to answer when the same input produces different output on each run.

Temperature reshapes the distribution. top-p truncates its tail — which is what actually stops the model reaching into the tens of thousands of tokens sitting at near-zero probability.

top-k keeps a fixed count regardless of confidence; top-p adapts — few tokens when the model is sure, many when it is not.

Act 5 Fluency ≠ Truth

## Nothing in that pipeline contains a notion of truth

The model was trained to produce likely text. Everything we have just taken apart explains why it hallucinates — and why the hallucination is fluent.
Asked for a citation, ungrounded

    Treatment-free remission ... minimum of
    5 years ... MR4.0 for at least 2 years.

    Source: European LeukemiaNet 2020
    recommendations, Section 6.4.
    PMID: 32127639

    Grade B recommendation based on the
    EURO-SKI and STIM2 trials.

Section number, grade and PMID: generated the same way as every other token.

There is no “I am now citing” mode inside the model. A citation is text, and text is what it produces.

It is not lying. It has no notion of a claim that could be false.

Why this one matters

The fabrication is not rare, not flagged, and not distinguishable from a real citation by reading it. You have to check.

Act 5 Fluency ≠ Truth

## The fix: change the task

Same model, same temperature, sources in the prompt

    Chronic phase throughout; any history of
    accelerated phase makes the patient
    ineligible [S01].
    TKI therapy for a minimum of 5 years [S01].
    MR4.0 maintained for at least 2 years [S01].
    Monthly transcripts for 6 months [S02].

    The provided sources do not contain a
    PubMed ID, so I cannot supply one.

The model did not become more truthful. We changed the task from recall a fact to read this and report what it says — which is a task it is genuinely good at.

Asked for a PMID that is not in the sources, it now says so instead of inventing one.

But grounding moves the failure, it does not remove it. Retrieve the wrong snippet and you get a confident answer citing the wrong source — which is worse, because it looks verified.

And Act 2 showed you exactly how fragile that retrieval step is.

Act 6 What this means for your work

## Where it earns its place

Extract structure from free text — verify a sample, measure the error rate
Change register — clinician ↔ patient language
Draft and summarise, with review
Semantic search over notes — check how much of the note it actually reads; negation will bite
Answer clinical questions from memory — ground it or don't ask it
Produce citations without retrieval — every PMID needs checking
The pattern: these models fail silently and fluently. Design the check before you design the system.
Act 6 What this means for your work

## Before PHI goes anywhere near this

Why our notes were synthetic

Real notes are personal data under GDPR. Sending them to a public API endpoint is a transfer. The Heidelberg lab endpoint used in the notebook is a local deployment — that is the reason it exists.

Access

MIMIC and n2c2 require a signed data use agreement. So does your institution's data. “It was on Hugging Face” is not a licence.

EU AI Act

An AI system informing clinical decisions is likely high-risk. That classification brings obligations on data governance, logging, human oversight and post-market monitoring — regardless of how good the demo looked.

The hard part of clinical AI was never the model. It is knowing what the system does when it is wrong, and being able to show that you knew.
Go deeper

## Where to go next

See a model in three dimensions
Bycroft’s LLM Visualization — a working GPT you can walk through, every matrix multiply visible. bbycroft.net/llm
Run one in your browser
Georgia Tech’s Transformer Explainer — a live GPT-2 you can type into, nothing to install. poloclub.github.io/transformer-explainer
The rigorous version
Jurafsky & Martin, Speech and Language Processing, 3rd ed. Free. Chapters 9–10 are today, done properly. web.stanford.edu/~jurafsky/slp3
Do it yourself
Everything here is reproducible from the workshop notebook — no coding needed, and it explains itself. llm_workshop.ipynb
If you write code
The same six ideas with the numpy exposed, plus five exercises and worked answers. llm_workshop_advanced.ipynb
Everything, on your phone
Slides, the no-install version and the notebook. sfb1709.jeffai.net
Open-source components: reveal.js (MIT) · GPT-2, Bio_ClinicalBERT (MIT) · all-MiniLM-L6-v2, PubMedBERT embeddings, Qwen2.5, BERT (Apache-2.0). Clinical text synthetic; guideline sources fictional. Full attributions in CITATIONS.md.