# Citations and attributions

Everything reused in this workshop, with its licence. Verified reachable and
correctly licensed at build time.

---

## Software

| Component | Licence | Use here |
|---|---|---|
| [reveal.js 5.1.0](https://revealjs.com) — Hakim El Hattab | MIT | Presentation framework. CSS and JS vendored and inlined into `slides/deck.html` so the deck runs offline. |
| [transformers](https://github.com/huggingface/transformers) — Hugging Face | Apache-2.0 | Tokenizers, GPT-2 inference, attention extraction. |
| [sentence-transformers](https://github.com/UKPLab/sentence-transformers) — Reimers & Gurevych | Apache-2.0 | Sentence embedding models in Act 2. |
| [PyTorch](https://pytorch.org) | BSD-3-Clause | Model execution. |
| [scikit-learn](https://scikit-learn.org) | BSD-3-Clause | PCA for the embedding projection. |
| [SciPy](https://scipy.org) | BSD-3-Clause | Orthogonal Procrustes alignment between the two projections. |
| [NumPy](https://numpy.org), [pandas](https://pandas.pydata.org), [Matplotlib](https://matplotlib.org) | BSD-3-Clause / BSD-3-Clause / PSF-based | Numerics and notebook figures. |
| [ipywidgets](https://github.com/jupyter-widgets/ipywidgets) | BSD-3-Clause | The sliders and text boxes in the beginner notebook. |
| [OpenAI Python SDK](https://github.com/openai/openai-python) | Apache-2.0 | Client for the OpenAI-compatible Heidelberg LiteLLM proxy. |

The GPT-2 byte-pair encoder in `slides/widgets.js` and `web/explore.js` is an
original implementation written for this workshop. It is verified against the
reference Hugging Face tokenizer — 10/10 exact token-ID matches across ASCII,
Unicode, leading whitespace, and cytogenetic notation. The algorithm itself is from Sennrich et
al. (below); no code was copied.

## Models

All obtained from the Hugging Face Hub. Licences confirmed via the Hub API.

| Model | Licence | Use here |
|---|---|---|
| [`openai-community/gpt2`](https://huggingface.co/openai-community/gpt2) | MIT | Tokenizer, attention patterns (Act 3), next-token logits (Act 4). |
| [`google-bert/bert-base-uncased`](https://huggingface.co/google-bert/bert-base-uncased) | Apache-2.0 | Tokenizer comparison (Act 1). |
| [`emilyalsentzer/Bio_ClinicalBERT`](https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT) | MIT | Clinical-domain tokenizer comparison (Act 1). |
| [`Qwen/Qwen2.5-0.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) | Apache-2.0 | Modern tokenizer comparison (Act 1). |
| [`sentence-transformers/all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | Apache-2.0 | General-purpose encoder (Act 2). |
| [`NeuML/pubmedbert-base-embeddings`](https://huggingface.co/NeuML/pubmedbert-base-embeddings) | Apache-2.0 | Biomedical encoder (Act 2). |

Act 5 uses `qwen-3.5-27b` and `nemotron-3-super` served through the Heidelberg
SAP proxy LiteLLM deployment. That is institutional infrastructure, not a
redistributable artefact; see the internal API documentation for access.

## Teaching resources referenced

Linked for further study. **No assets, code, or text were copied from any of
these** — they are recommendations, listed here so the recommendation is
attributable.

| Resource | Licence | Where |
|---|---|---|
| [LLM Visualization](https://bbycroft.net/llm) — Brendan Bycroft ([source](https://github.com/bbycroft/llm-viz)) | MIT | Recommended in Act 6 and in the notebook as the 3D deep-dive. |
| [Transformer Explainer](https://poloclub.github.io/transformer-explainer/) — Cho, Wang et al., Georgia Tech Polo Club | MIT | Recommended for self-study. |
| [BertViz](https://github.com/jessevig/bertviz) — Jesse Vig | Apache-2.0 | Named in Act 3 as the full-featured version of the attention view built there. |
| [The Illustrated Transformer](https://jalammar.github.io/illustrated-transformer/) — Jay Alammar | — (linked only) | Recommended written explainer. |
| [Speech and Language Processing, 3rd ed.](https://web.stanford.edu/~jurafsky/slp3/) — Jurafsky & Martin | Free online | Recommended rigorous treatment; chapters 9–10. |

## Figures taken from published literature

The two opening slides are the only figures in this deck that are **not** our own
model runs. Both are labelled as such on the slide.

| Figure | Values | Source |
|---|---|---|
| Model-scale timeline (slide 2) | GPT-1 117M, GPT-2 1.5B, GPT-3 175B; GPT-4 and current-generation counts are published *estimates* — the labs stopped disclosing them | Radford et al. 2018/2019; Brown et al. 2020; press and industry estimates thereafter |
| MedQA accuracy (slide 3) | Med-PaLM 67.6%, Med-PaLM 2 86.5% | Singhal et al., *Towards Expert-Level Medical Question Answering with Large Language Models*, [arXiv:2305.09617](https://arxiv.org/abs/2305.09617) |
| MedQA accuracy (slide 3) | GPT-4 81.4%, GPT-4 + Medprompt 90.2% | Nori et al., *Can Generalist Foundation Models Outcompete Special-Purpose Tuning?*, [arXiv:2311.16452](https://arxiv.org/abs/2311.16452) |

The advertised-versus-effective context-window gap referenced on slide 7 is a
widely reproduced finding rather than a single result; the deck states it
qualitatively and claims no specific number.

## Papers

- Vaswani, A. et al. (2017). *Attention Is All You Need.* [arXiv:1706.03762](https://arxiv.org/abs/1706.03762)
- Sennrich, R., Haddow, B., & Birch, A. (2016). *Neural Machine Translation of Rare Words with Subword Units.* [arXiv:1508.07909](https://arxiv.org/abs/1508.07909) — the BPE algorithm implemented in the tokenizer widget.
- Radford, A. et al. (2019). *Language Models are Unsupervised Multitask Learners.* (GPT-2)
- Reimers, N. & Gurevych, I. (2019). *Sentence-BERT.* [arXiv:1908.10084](https://arxiv.org/abs/1908.10084)
- Alsentzer, E. et al. (2019). *Publicly Available Clinical BERT Embeddings.* [arXiv:1904.03323](https://arxiv.org/abs/1904.03323)
- Gu, Y. et al. (2021). *Domain-Specific Language Model Pretraining for Biomedical NLP.* [arXiv:2007.15779](https://arxiv.org/abs/2007.15779) — PubMedBERT.
- Singhal, K. et al. (2023). *Towards Expert-Level Medical Question Answering with Large Language Models.* [arXiv:2305.09617](https://arxiv.org/abs/2305.09617)
- Nori, H. et al. (2023). *Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine.* [arXiv:2311.16452](https://arxiv.org/abs/2311.16452)
- Liu, N. F. et al. (2023). *Lost in the Middle: How Language Models Use Long Contexts.* [arXiv:2307.03172](https://arxiv.org/abs/2307.03172)
- Holtzman, A. et al. (2020). *The Curious Case of Neural Text Degeneration.* [arXiv:1904.09751](https://arxiv.org/abs/1904.09751) — nucleus (top-p) sampling, implemented in Act 4.

---

## Data: what we did *not* use, and why

**All clinical text in this workshop is synthetic**, written specifically for
this session. See the `_provenance` block at the top of
`data/synthetic_notes.json` and `data/guideline_snippets.json`.

The "guideline" sources are **deliberately fictional**. This is pedagogical, not
lazy: Act 5 demonstrates a model fabricating a citation and then fixes it with
retrieval. If real citations were mixed in, attendees could not tell which
output was fabricated and which was retrieved.

The following real corpora are **cited as pointers only and are not
redistributed here**, because they require credentialed access or a signed data
use agreement:

| Corpus | Access condition |
|---|---|
| [MIMIC-III / MIMIC-IV](https://physionet.org/content/mimiciv/) | PhysioNet credentialing + DUA |
| n2c2 / i2b2 clinical NLP datasets — see [i2b2.org](https://www.i2b2.org/), access now via the Harvard DBMI Data Portal | DUA |
| [MedQA](https://huggingface.co/datasets/bigbio/med_qa) | MIT — usable, simply not needed here |
| [PubMedQA](https://huggingface.co/datasets/bigbio/pubmed_qa) | MIT |
| [MedMCQA](https://huggingface.co/datasets/openlifescienceai/medmcqa) | Apache-2.0 |
| [MedCalc-Bench](https://huggingface.co/datasets/ncbi/MedCalc-Bench) | CC-BY-SA 4.0, non-commercial |

If you want to extend this workshop with real data, start with MedQA or
PubMedQA — both are permissively licensed and need no credentialing.

---

## This workshop

Materials in this repository that are not listed above — the synthetic clinical
corpus, the slide deck design, the widget implementations, the notebooks, the
companion web page, and the supporting Python — were written for the SFB1709
Summer School,
*Cellular Plasticity in Myeloid Malignancies: From Mechanisms to Therapies*.
