{ "cells": [ { "cell_type": "markdown", "id": "cell-000", "metadata": {}, "source": [ "# How Large Language Models Work\n", "\n", "**Summer School SFB1709** — *Cellular Plasticity in Myeloid Malignancies: From Mechanisms to Therapies*\n", "\n", "---\n", "\n", "### You do not need to know how to code to use this notebook.\n", "\n", "Every grey box below is a piece of code. You never have to write or change any of\n", "it. To run one, click on it and press **Shift + Enter** — or click the ▶ button on\n", "its left. The interesting part is what appears *underneath*.\n", "\n", "Where there is something to play with, you get a text box or a slider. No typing\n", "of code, ever.\n", "\n", "### Three ways to use this\n", "\n", "| If you are... | Do this |\n", "|---|---|\n", "| **Watching the talk** | Nothing. Just look at the pictures as they come up. |\n", "| **Following on a laptop** | Run each cell as we reach it. Type your own words into the boxes. |\n", "| **Reading this later** | Run everything top to bottom. The text explains what each result means — you do not need the talk. |\n", "\n", "### What you should end up believing\n", "\n", "That a language model is a machine which, shown some text, guesses what comes\n", "next — and that everything else about it, the fluency, the apparent understanding,\n", "the confident wrong answers, follows from that one fact.\n", "\n", "---\n", "\n", "### Two promises about the content\n", "\n", "- **Every patient here is invented.** Ten discharge summaries were written for this\n", " session. No real records, no patient data. The \"guidelines\" we use later are\n", " fictional on purpose — you will see why in Part 6.\n", "- **Nothing here is medical advice**, and none of it should inform care." ] }, { "cell_type": "markdown", "id": "cell-001", "metadata": {}, "source": [ "---\n", "## Setup — run this once\n", "\n", "This takes about a minute the first time. It downloads the models we will look at.\n", "You will see some progress bars; that is normal." ] }, { "cell_type": "code", "id": "cell-002", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# --- Setup: run me first --------------------------------------------------\n", "import importlib.util, io, os, subprocess, sys, urllib.request, zipfile\n", "from pathlib import Path\n", "\n", "# Install anything that is missing (Colab already has most of it).\n", "missing = [p for p in (\"transformers\", \"sentence_transformers\", \"openai\", \"pandas\")\n", " if importlib.util.find_spec(p) is None]\n", "if missing:\n", " print(f\"Installing {', '.join(missing)} — about a minute...\")\n", " subprocess.run([sys.executable, \"-m\", \"pip\", \"install\", \"-q\", *missing], check=True)\n", "\n", "# Find the workshop files, or fetch them (36 KB). Nothing to configure: they\n", "# come from the workshop's own site, so this works in Colab, on your laptop,\n", "# and anywhere else with an internet connection.\n", "MATERIALS = \"https://sfb1709.jeffai.net/workshop-files.zip\"\n", "\n", "def find_files():\n", " here = Path.cwd()\n", " for folder in [here, *here.parents, here / \"llm_workshop\",\n", " Path(\"/content/llm_workshop\")]:\n", " if (folder / \"data\" / \"synthetic_notes.json\").is_file():\n", " return folder\n", " return None\n", "\n", "REPO = find_files()\n", "if REPO is None:\n", " print(\"Downloading the workshop files ...\")\n", " try:\n", " with urllib.request.urlopen(MATERIALS, timeout=90) as response:\n", " zipfile.ZipFile(io.BytesIO(response.read())).extractall(Path.cwd())\n", " except Exception as problem:\n", " raise SystemExit(\n", " \"Could not download the workshop files: \" + str(problem)\n", " + \". Check your internet connection, or download \" + MATERIALS\n", " + \" by hand and unzip it next to this notebook.\"\n", " )\n", " REPO = find_files()\n", "if REPO is None:\n", " raise SystemExit(\"Downloaded the files but could not locate them — please tell the presenter.\")\n", "\n", "sys.path.insert(0, str(REPO / \"src\"))\n", "os.chdir(REPO)\n", "\n", "import teaching as t # the pictures and tables\n", "import interactive as i # the boxes and sliders\n", "\n", "print()\n", "print(\"Ready. Everything below will work now.\")" ] }, { "cell_type": "markdown", "id": "cell-003", "metadata": {}, "source": [ "---\n", "# Part 1 · The one idea\n", "\n", "Everything a language model does comes from a single ability: **given some text,\n", "it guesses what word comes next.**\n", "\n", "Not \"understands\". Not \"looks up\". Guesses — and it produces not one guess but a\n", "score for *every* word it knows, all fifty thousand of them.\n", "\n", "Run the cell below. We give the model an unfinished sentence and ask what it\n", "thinks comes next." ] }, { "cell_type": "code", "id": "cell-004", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.predict_next_word(\"The patient was admitted to hospital with acute\")" ] }, { "cell_type": "markdown", "id": "cell-005", "metadata": {}, "source": [ "### What just happened\n", "\n", "The model read that unfinished sentence and produced a **probability for every\n", "word in its vocabulary**. The bar chart shows the ten it liked most.\n", "\n", "Notice that it is not certain. It spreads its bet across several plausible\n", "continuations, and each has a percentage. That spread is the model's actual\n", "output — a single word only appears when something *chooses* one from this list,\n", "which is what Part 5 is about.\n", "\n", "Now stack that up: pick a word, add it to the sentence, and ask again. And again.\n", "Do it three hundred times and you have a paragraph that reads like it was written\n", "by someone who knows what they are talking about.\n", "\n", "**That is the whole machine.** Everything from here is detail about how it makes\n", "these guesses well — and about the ways that go wrong." ] }, { "cell_type": "markdown", "id": "cell-006", "metadata": {}, "source": [ "### The text we will use throughout\n", "\n", "Let us look at what we are working with. These are the ten invented discharge\n", "summaries — one per row." ] }, { "cell_type": "code", "id": "cell-007", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.list_notes()" ] }, { "cell_type": "code", "id": "cell-008", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "i.note_browser()" ] }, { "cell_type": "markdown", "id": "cell-009", "metadata": {}, "source": [ "These are written to look like real myeloid haematology notes: an AML, an MDS, a\n", "CML, a myelofibrosis, a transplant complication, and so on. Every name, date and\n", "number is invented." ] }, { "cell_type": "markdown", "id": "cell-010", "metadata": {}, "source": [ "---\n", "# Part 2 · The model cannot see words\n", "\n", "Here is the first surprise. The model has never seen a word in its life.\n", "\n", "Before anything happens, your text is chopped into pieces called **tokens**. The\n", "model has a fixed list of about 50,000 of them and can only work with those. A\n", "common word like `the` is one token. A rare one gets broken into fragments.\n", "\n", "Run this on an ordinary sentence." ] }, { "cell_type": "code", "id": "cell-011", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.show_tokens(\"The patient has acute myeloid leukaemia.\")" ] }, { "cell_type": "markdown", "id": "cell-012", "metadata": {}, "source": [ "Each coloured box is one token. Mostly whole words — this is everyday English, so\n", "the pieces line up with how we read.\n", "\n", "Now the same thing on something from your world." ] }, { "cell_type": "code", "id": "cell-013", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.show_tokens(\"t(9;22)(q34;q11)\")" ] }, { "cell_type": "markdown", "id": "cell-014", "metadata": {}, "source": [ "### What you should notice\n", "\n", "The Philadelphia translocation — sixteen characters, one idea, something every\n", "haematologist reads as a single unit — costs the model **twelve separate tokens**.\n", "It sees a pile of brackets, digits and semicolons.\n", "\n", "Why? These token lists are built by counting what appears in ordinary internet\n", "text. `the` earns its own token because it appears constantly. `t(9;22)(q34;q11)`\n", "does not appear often enough on the open web to earn one, so the model has to\n", "spell it out almost character by character.\n", "\n", "**The consequence is practical, not philosophical.** Tokens are what you pay for,\n", "what fills the model's limited memory, and what it has to reason over. Your\n", "notation is roughly three to four times more expensive than ordinary prose, and\n", "the concept you care about arrives fragmented.\n", "\n", "Try it yourself — a gene, a drug, a phrase from your own notes:" ] }, { "cell_type": "code", "id": "cell-015", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "i.tokenizer_box()" ] }, { "cell_type": "code", "id": "cell-016", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.token_cost_table()" ] }, { "cell_type": "markdown", "id": "cell-017", "metadata": {}, "source": [ "Green rows are cheap, red rows expensive. The pattern is not word length — it is\n", "**punctuation and rarity**. `midostaurin` is a long word but the model has seen\n", "it; `del(5q)` is short but full of brackets.\n", "\n", "### Why this matters for real records" ] }, { "cell_type": "code", "id": "cell-018", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.context_math()" ] }, { "cell_type": "markdown", "id": "cell-019", "metadata": {}, "source": [ "A model has a fixed amount it can hold at once. Even a large one, holding 128,000\n", "tokens, fits only a couple of hundred of our short notes — and a real oncology\n", "record, with years of clinic letters and results, is far bigger.\n", "\n", "So **something has to decide what goes into that window.** In practice that\n", "selection step, not the model, is what determines the answer you get. Keep that\n", "in mind for Part 6." ] }, { "cell_type": "markdown", "id": "cell-020", "metadata": {}, "source": [ "---\n", "# Part 3 · Turning meaning into distance\n", "\n", "To compare two pieces of text, a model turns each into a long list of numbers —\n", "a point in space. Similar meanings are supposed to land near each other.\n", "\n", "This is what powers \"find similar patients\", semantic search, and every\n", "retrieval system you have been shown. So it is worth checking whether it works.\n", "\n", "We will use two models. One is **general-purpose** — the default in almost every\n", "tutorial. The other was trained on **biomedical literature**.\n", "\n", "First, an easy one. `AML` and `acute myeloid leukaemia` are the same disease." ] }, { "cell_type": "code", "id": "cell-021", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.compare_meaning(\"AML\", \"acute myeloid leukaemia\", expect=\"same\")" ] }, { "cell_type": "markdown", "id": "cell-022", "metadata": {}, "source": [ "### What you should notice\n", "\n", "The biomedical model scores this around **0.68** — it knows these are related.\n", "\n", "The general-purpose model scores about **0.18**. It does *not* know that AML is\n", "acute myeloid leukaemia. It has mostly encountered \"AML\" as an abbreviation for\n", "other things entirely.\n", "\n", "If you built a patient-similarity search on the general model — which is the\n", "default choice in most tutorials — **every note that used the abbreviation would\n", "quietly drop out of your results.** No error. No warning. Just a smaller answer\n", "than you expected.\n", "\n", "That is the argument for using domain-specific models, and it took one line to\n", "demonstrate.\n", "\n", "### Now the harder problem\n", "\n", "Both models were fine on that. Here is one neither of them handles." ] }, { "cell_type": "code", "id": "cell-023", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.compare_meaning(\"no FLT3-ITD mutation detected\", \"FLT3-ITD mutation detected\", expect=\"different\")" ] }, { "cell_type": "markdown", "id": "cell-024", "metadata": {}, "source": [ "### Read that again\n", "\n", "Those two statements are **clinically opposite**. One patient gets a FLT3\n", "inhibitor. The other does not.\n", "\n", "The general model scores them **0.985** — as near identical as makes no\n", "difference. The biomedical model manages 0.946, which is not a meaningful\n", "improvement.\n", "\n", "Why does this happen? These models measure something close to *what is this text\n", "about*. Both sentences are about FLT3-ITD. The word \"no\" is one small token among\n", "nine, and nothing in the way these models were trained taught them that it flips\n", "the meaning.\n", "\n", "**The practical consequence:** any system that finds clinical text by similarity\n", "will happily return the patient who *does not* have the thing you searched for.\n", "This is the single most common way clinical text-search projects fail, and it\n", "fails silently.\n", "\n", "Try to break it yourself. Pick two phrases and say what you think — the model\n", "will tell you whether it agrees:" ] }, { "cell_type": "code", "id": "cell-025", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "i.meaning_explorer()" ] }, { "cell_type": "code", "id": "cell-026", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.meaning_table()" ] }, { "cell_type": "markdown", "id": "cell-027", "metadata": {}, "source": [ "### One more thing, and this one is a genuine bug you may already have\n", "\n", "Before blaming the models for poor results, it is worth checking something much\n", "more basic: **how much of each note did they actually read?**" ] }, { "cell_type": "code", "id": "cell-028", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.how_much_did_it_read()" ] }, { "cell_type": "markdown", "id": "cell-029", "metadata": {}, "source": [ "### What you should notice\n", "\n", "The popular general-purpose model accepts **256 tokens**. Our notes average over\n", "500. It read about the first half of each one and **threw the rest away without\n", "telling anyone** — no error, no warning, nothing in the result to indicate it\n", "happened.\n", "\n", "The half it discarded contains the hospital course and the discharge medications.\n", "Which is to say: the part with the actual treatment in it.\n", "\n", "If you are using this kind of similarity search on clinical documents, check this\n", "first. It is the cheapest and most common fix available.\n", "\n", "Here is what similarity search gives us on the notes:" ] }, { "cell_type": "code", "id": "cell-030", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.find_similar_note('N01')" ] }, { "cell_type": "markdown", "id": "cell-031", "metadata": {}, "source": [ "Reasonable, but not reliable — and the notes all share a huge amount of\n", "vocabulary. Every one of them mentions marrow, blasts, counts and cycles.\n", "**Being about similar things is not the same as being clinically similar**, and\n", "no amount of model-swapping fixes that on its own." ] }, { "cell_type": "markdown", "id": "cell-032", "metadata": {}, "source": [ "---\n", "# Part 4 · How the model reads context\n", "\n", "Part 3 showed that squashing a sentence into one point loses the word \"no\". So how\n", "do these models handle context at all?\n", "\n", "The mechanism is called **attention**. As it processes each word, the model looks\n", "back at the earlier words and decides how much each one matters. The word\n", "`disease` can look back and notice that `No` appeared earlier.\n", "\n", "The picture below shows one of these patterns. Read a row left to right: it shows\n", "where that word looked." ] }, { "cell_type": "code", "id": "cell-033", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.show_attention()" ] }, { "cell_type": "markdown", "id": "cell-034", "metadata": {}, "source": [ "### What you should notice\n", "\n", "Two things.\n", "\n", "**The empty top-right triangle.** A word can only look *backwards*, never forwards.\n", "That is what makes the model able to write left to right, one word at a time — it\n", "is the same constraint as in Part 1.\n", "\n", "**The information is there.** Some of these patterns do link a word back to the\n", "\"No\" in front of it. Attention can represent negation perfectly well.\n", "\n", "Which raises the obvious question: if the model *can* see the negation, why did\n", "Part 3 lose it? Because turning a whole sentence into a single point **averages\n", "all these word-by-word signals together**, and the average washes the negation\n", "out. The failure was not in the reading. It was in the squashing.\n", "\n", "That distinction is practical: use similarity search to find roughly relevant\n", "documents, but never to decide a question that turns on one word.\n", "\n", "The model has 144 of these patterns. Have a look around if you like:" ] }, { "cell_type": "code", "id": "cell-035", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "i.attention_explorer()" ] }, { "cell_type": "markdown", "id": "cell-036", "metadata": {}, "source": [ "---\n", "# Part 5 · Choosing the next word\n", "\n", "Back to Part 1. The model gives us a *list* of candidate words with scores. Something\n", "has to pick one — and that something is a setting you control, called\n", "**temperature**.\n", "\n", "Here is the same question at four settings." ] }, { "cell_type": "code", "id": "cell-037", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.temperature_comparison()" ] }, { "cell_type": "markdown", "id": "cell-038", "metadata": {}, "source": [ "### What you should notice\n", "\n", "Nothing about the model changed between those four pictures. Same model, same\n", "question. Only the dial moved.\n", "\n", "- **Low temperature** — the model always picks its favourite. Ask twice, get the\n", " same answer twice.\n", "- **High temperature** — unlikely words become live options. More varied, and more\n", " likely to be wrong.\n", "\n", "Have a go:" ] }, { "cell_type": "code", "id": "cell-039", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "i.temperature_slider()" ] }, { "cell_type": "markdown", "id": "cell-040", "metadata": {}, "source": [ "### Why this is a governance question, not a style preference\n", "\n", "If you are pulling a drug dose out of a note, you want temperature at zero: the\n", "same input should give the same output, every time, and you should be able to\n", "show why.\n", "\n", "The moment temperature is above zero, **the same input can produce different\n", "output on different runs.** \"Why did the system say that?\" becomes a question you\n", "cannot fully answer after the fact — which matters a great deal if anyone ever\n", "audits the thing.\n", "\n", "Most tools default to a middle setting. It is worth knowing that, and knowing it\n", "is yours to change." ] }, { "cell_type": "markdown", "id": "cell-041", "metadata": {}, "source": [ "---\n", "# Part 6 · Why it makes things up — and what actually helps\n", "\n", "Everything so far explains the problem. The model was trained to produce **likely\n", "text**. Nothing anywhere in it contains a notion of *true*.\n", "\n", "So far we have used GPT-2, a small model from 2019, because it is simple enough to\n", "look inside. Now we switch to a proper modern one.\n", "\n", "The cell below connects to our local lab model. If it cannot reach it — which may\n", "happen on conference wifi — it falls back to prepared answers and **tells you\n", "exactly what it is showing you**, including whether the text came from a live\n", "model or was written in advance. Either way the demonstration works, and you are\n", "never shown a prepared answer dressed up as a live one." ] }, { "cell_type": "code", "id": "cell-042", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "model = t.connect()" ] }, { "cell_type": "markdown", "id": "cell-043", "metadata": {}, "source": [ "### The trap\n", "\n", "We are going to ask a real clinical question and insist on a citation we could\n", "check." ] }, { "cell_type": "code", "id": "cell-044", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.ask(model, \"citation_ungrounded\", show_question=True)" ] }, { "cell_type": "markdown", "id": "cell-045", "metadata": {}, "source": [ "### What you should notice\n", "\n", "Look at what came back: a clear answer, a named guideline, a section number, a\n", "recommendation grade, and a **PubMed ID**.\n", "\n", "The section number is invented. The grade is invented. The PMID is an\n", "eight-digit number that looks exactly like a real one and was produced the same\n", "way every other word was produced — by guessing a likely continuation.\n", "\n", "There is no \"now I am citing\" mode inside the model. A citation is just text, and\n", "text is what it makes. **It is not lying to you.** It has no concept of a claim\n", "that could be false.\n", "\n", "This is the single most important thing to take away if you plan to use these\n", "tools for literature work. The fabrication is not rare, it is not flagged, and\n", "**you cannot tell by reading it**.\n", "\n", "### The fix\n", "\n", "Now the same question — same model, same settings — with one change: we paste the\n", "relevant source text into the question and tell it to use only that." ] }, { "cell_type": "code", "id": "cell-046", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.ask(model, \"citation_grounded\")" ] }, { "cell_type": "markdown", "id": "cell-047", "metadata": {}, "source": [ "### What changed\n", "\n", "Every claim now carries a source tag you can check. The numbers come from the text\n", "we supplied rather than from the model's memory. And when asked for a PMID that\n", "is not in the sources, **it says so instead of inventing one**.\n", "\n", "The model did not become more honest. We changed the job from *remember this* to\n", "*read this and tell me what it says* — and the second job is one it is genuinely\n", "good at.\n", "\n", "This is what people mean by \"RAG\" or \"grounding\", stripped of jargon: find the\n", "relevant text, put it in front of the model, tell it to stick to it.\n", "\n", "**But be careful about how much comfort to take from this.** Grounding moves the\n", "problem rather than removing it. If the search step fetches the wrong document,\n", "you now get a confident answer citing the wrong source — which is arguably worse,\n", "because it looks verified. And Part 3 showed you exactly how fragile that search\n", "step is.\n", "\n", "### What these models are genuinely good at\n", "\n", "Enough failure. Here is the thing they do well: pulling structured information out\n", "of free text." ] }, { "cell_type": "code", "id": "cell-048", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.ask(model, \"extract_structured\")" ] }, { "cell_type": "markdown", "id": "cell-049", "metadata": {}, "source": [ "### Now check it against the note\n", "\n", "That output is clean, well-organised and confident. **It also contains an error.**\n", "\n", "Have a look at the note it came from and see whether you can spot it before\n", "reading on." ] }, { "cell_type": "code", "id": "cell-050", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t.show_note('N01', chars=2000)" ] }, { "cell_type": "markdown", "id": "cell-051", "metadata": {}, "source": [ "
\n", "Click here for the answer\n", "

The extraction lists allopurinol as a discharge medication. The note says\n", "the opposite — allopurinol discontinued.

\n", "

A drug that was stopped has been carried forward into the take-home\n", "list. In a real medication reconciliation workflow that is exactly the kind of\n", "error that causes harm, and notice what it is not: not garbled, not\n", "malformed, not flagged as uncertain. Clean, plausible, and wrong.

\n", "
\n", "\n", "The lesson is the one that runs through the whole session: **these systems fail\n", "fluently.** The output looks equally good whether it is right or wrong, so you\n", "cannot use \"does it look right\" as your check. You need a real one." ] }, { "cell_type": "markdown", "id": "cell-052", "metadata": {}, "source": [ "---\n", "# Part 7 · What to take away\n", "\n", "### The machine, in six sentences\n", "\n", "1. A language model guesses the next word, over and over.\n", "2. It cannot see words — only **tokens**, and your notation breaks into far more of\n", " them than ordinary English does.\n", "3. It compares meaning by **distance between points**, which captures topic well\n", " and negation badly.\n", "4. **Attention** lets it read context properly; squashing a sentence into one point\n", " is what throws that away.\n", "5. A **dial you control** decides how adventurous each word choice is.\n", "6. Nowhere in any of that is there a notion of **truth**.\n", "\n", "### Where it is worth using\n", "\n", "| Task | Verdict |\n", "|---|---|\n", "| Pulling structured data out of free text | **Good** — but check a sample and measure how often it is wrong |\n", "| Rewriting for a different reader (clinician ↔ patient) | **Good** — low risk, saves real time |\n", "| Drafting and summarising | **Good, with review** |\n", "| Searching notes by meaning | **Careful** — check how much of each note it actually reads; negation will bite you |\n", "| Answering clinical questions from memory | **No** — give it sources or do not ask |\n", "| Producing references | **No**, unless it is quoting sources you supplied |\n", "\n", "### Before you put real patient data anywhere near this\n", "\n", "- Our notes were invented deliberately. Real notes are personal data under GDPR,\n", " and sending them to a commercial API is a data transfer.\n", "- The lab model we used in Part 6 runs **locally**. That is the entire reason it\n", " exists.\n", "- Public research datasets like MIMIC require a signed agreement. So does your own\n", " institution's data.\n", "- Under the EU AI Act, a system informing clinical decisions is likely\n", " **high-risk**, which brings real obligations regardless of how good a demo looks.\n", "\n", "### If you want to go further\n", "\n", "- **[LLM Visualization](https://bbycroft.net/llm)** — a working model drawn in 3D\n", " that you can walk through. Genuinely worth twenty minutes.\n", "- **[Transformer Explainer](https://poloclub.github.io/transformer-explainer/)** —\n", " a live model in your browser, nothing to install.\n", "- **`llm_workshop_advanced.ipynb`** — in this same folder. The same six ideas with\n", " the code exposed, for anyone who wants to modify it. `solutions.ipynb` has worked\n", " answers to its exercises.\n", "\n", "---\n", "\n", "*All clinical text in this notebook is synthetic. Full attributions and licences\n", "are in `CITATIONS.md`.*" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python", "version": "3.12" }, "colab": { "provenance": [], "toc_visible": true } }, "nbformat": 4, "nbformat_minor": 5 }