Reading Guide - Historical Significance vs Current Relevance¶
Not every important paper is a paper you still need to read. This guide separates the two: how much a paper mattered when it landed, and how much it matters to your work today. Where those two diverge, it says which way to lean and what to read instead.
It is a prioritisation guide, not a catalogue. Every one of the 107 papers carries a relevance badge in BROWSE.md, a one-line contribution in QUICK_REFERENCE.md, and a full entry in INDEX.md. This page covers the essentials plus the cases where the two axes pull apart.
📊 Priority Matrix¶
Historical = how much it changed the field at the time. Current = how much you lose today by not having read it.
Foundations you cannot skip¶
| Paper | Year | Historical | Current | Priority | Read For |
|---|---|---|---|---|---|
| Transformer | 2017 | 🔥🔥🔥🔥🔥 | 🔥🔥🔥🔥🔥 | CRITICAL | The architecture under everything |
| Scaling Laws | 2020 | 🔥🔥🔥🔥🔥 | 🔥🔥🔥🔥🔥 | CRITICAL | Why compute budgets look like they do |
| Chinchilla | 2022 | 🔥🔥🔥🔥 | 🔥🔥🔥🔥🔥 | CRITICAL | The data/parameter ratio everyone now uses |
| InstructGPT | 2022 | 🔥🔥🔥🔥🔥 | 🔥🔥🔥🔥🔥 | CRITICAL | How a base model becomes an assistant |
| Chain-of-Thought | 2022 | 🔥🔥🔥🔥 | 🔥🔥🔥🔥🔥 | CRITICAL | The root of every reasoning method |
| RAG | 2020 | 🔥🔥🔥🔥 | 🔥🔥🔥🔥🔥 | CRITICAL | The most-deployed LLM pattern there is |
| LoRA | 2021 | 🔥🔥🔥🔥 | 🔥🔥🔥🔥🔥 | CRITICAL | How fine-tuning is actually done |
| GPT-3 | 2020 | 🔥🔥🔥🔥🔥 | 🔥🔥🔥🔥 | HIGH | Where prompting replaced fine-tuning |
The modern layer (2023-2026)¶
These did not exist when most "essential papers" lists were written, and they are where current practice actually lives.
| Paper | Year | Historical | Current | Priority | Read For |
|---|---|---|---|---|---|
| DeepSeek-R1 | 2025 | 🔥🔥🔥🔥 | 🔥🔥🔥🔥🔥 | CRITICAL | Reasoning training, fully disclosed |
| RLVR | 2024-25 | 🔥🔥🔥🔥 | 🔥🔥🔥🔥🔥 | CRITICAL | The signal reasoning models train on |
| GRPO | 2024 | 🔥🔥🔥 | 🔥🔥🔥🔥🔥 | CRITICAL | RLHF without the critic model |
| Test-Time Compute | 2024 | 🔥🔥🔥🔥 | 🔥🔥🔥🔥🔥 | CRITICAL | Why "think longer" beats "train bigger" |
| Mixtral / MoE | 2024 | 🔥🔥🔥🔥 | 🔥🔥🔥🔥🔥 | CRITICAL | The frontier architecture default |
| DPO | 2023 | 🔥🔥🔥🔥 | 🔥🔥🔥🔥🔥 | CRITICAL | Alignment without an RL loop |
| RoPE | 2021 | 🔥🔥🔥 | 🔥🔥🔥🔥🔥 | CRITICAL | Positions, in nearly every LLM you use |
| FlashAttention | 2022 | 🔥🔥🔥🔥 | 🔥🔥🔥🔥🔥 | CRITICAL | Why long context is affordable |
| PagedAttention / vLLM | 2023 | 🔥🔥🔥 | 🔥🔥🔥🔥🔥 | HIGH | How models are actually served |
| MCP | 2024 | 🔥🔥🔥 | 🔥🔥🔥🔥🔥 | HIGH | How tools get wired to models now |
| ReAct | 2023 | 🔥🔥🔥🔥 | 🔥🔥🔥🔥 | HIGH | The agent loop, in its original form |
| GQA | 2023 | 🔥🔥🔥 | 🔥🔥🔥🔥 | HIGH | KV-cache cost in production |
| GPTQ & AWQ | 2022-23 | 🔥🔥🔥 | 🔥🔥🔥🔥 | HIGH | Why local inference is possible |
| Dense Retrieval | 2019-20 | 🔥🔥🔥 | 🔥🔥🔥🔥 | HIGH | The layer under every RAG system |
| SWE-bench | 2023 | 🔥🔥🔥 | 🔥🔥🔥🔥 | HIGH | What coding-agent claims mean |
| Sparse Autoencoders | 2022-24 | 🔥🔥🔥🔥 | 🔥🔥🔥🔥 | HIGH | The state of interpretability |
Where the two axes diverge¶
Read these for understanding, not for practice. The "instead" column is what to spend the time on.
| Paper | Year | Historical | Current | Priority | Instead |
|---|---|---|---|---|---|
| BERT | 2018 | 🔥🔥🔥🔥🔥 | 🔥🔥 | MEDIUM | Decoder-only models; Dense Retrieval for embeddings |
| GANs | 2014 | 🔥🔥🔥🔥🔥 | 🔥🔥 | MEDIUM | Stable Diffusion, Flow Matching |
| Word2Vec | 2013 | 🔥🔥🔥🔥🔥 | 🔥🔥 | MEDIUM | Dense Retrieval |
| Seq2Seq | 2014 | 🔥🔥🔥🔥 | 🔥🔥 | MEDIUM | Transformer |
| GPT-1 | 2018 | 🔥🔥🔥🔥 | 🔥🔥 | MEDIUM | GPT-3 |
| GPT-2 | 2019 | 🔥🔥🔥🔥 | 🔥🔥 | MEDIUM | GPT-3 |
| DDPM | 2020 | 🔥🔥🔥🔥🔥 | 🔥🔥🔥 | MEDIUM | Read if you need the maths, not to use diffusion |
| PPO | 2017 | 🔥🔥🔥🔥🔥 | 🔥🔥🔥 | MEDIUM | DPO, GRPO unless you implement RLHF |
| PaLM | 2022 | 🔥🔥🔥🔥 | 🔥🔥 | MEDIUM | MoE models; dense scaling stopped here |
🎯 What Changed, and Why¶
The three cases readers ask about most.
BERT-style encoders¶
- 2018-2020: encoders dominated NLP; fine-tune BERT for every task.
- Today: decoder-only models do understanding and generation, so the split disappeared.
- Still used for: embeddings and reranking - Sentence-BERT and ColBERT, covered in Dense Retrieval.
- Read BERT if: you work on retrieval, or maintain a pre-2021 NLP system.
GANs¶
- 2014-2020: the default for image synthesis.
- Today: diffusion and flow matching win on stability, controllability and quality.
- Still used for: real-time generation and super-resolution, where a single forward pass matters. The adversarial loss also survives inside VQ-GAN, which is very much still in use.
- Read GANs if: you need the adversarial idea itself; otherwise read the diffusion line.
Dense scaling¶
- 2020-2022: make the dense model bigger - GPT-3 at 175B, PaLM at 540B.
- Today: sparse MoE gives more capacity per unit of inference cost, and Chinchilla showed the big dense models were under-trained anyway. Frontier models activate a fraction of their parameters.
- Read PaLM if: you want the high-water mark of the dense era; otherwise read Mixtral and DeepSeek-V3.
📈 What's Increasingly Important¶
Reasoning as a training target, not a prompt. Chain-of-thought was a prompting trick in 2022. By 2025 it is what the model is trained to do: STaR → Process Reward Models → Test-Time Compute → o1 → RLVR / DeepSeek-R1.
Alignment without the RL machinery. DPO removed the reward model, KTO removed the need for paired data, and GRPO removed the critic. Each step made alignment cheaper to run.
Inference economics. Serving cost now drives architecture: GQA, PagedAttention, Speculative Decoding, quantisation, and MoE all exist to make the forward pass cheaper.
Agents and tool use. ReAct → Reflexion → Generative Agents → Voyager, with MCP as the integration layer and SWE-bench as the scoreboard.
Interpretability and safety with teeth. Sparse Autoencoders made features readable and steerable; Sleeper Agents showed safety fine-tuning can leave a backdoor intact.
Everything outside language. The same toolkit now does biology (AlphaFold 2, AlphaFold 3, ESM-2), mathematics (AlphaGeometry), algorithm design (AlphaEvolve) and world modelling (Genie, DreamerV3).
📉 What's Becoming Less Relevant¶
- Encoder-only architectures - outside embeddings and reranking.
- GAN training tricks - the failure modes they solved mostly don't arise in diffusion.
- Separate vision and language stacks - ViT and CLIP collapsed them into one.
- Prompt engineering as a discipline - DALL-E 3 and instruction-tuned LLMs absorbed most of it.
- Absolute positional embeddings - RoPE won.
- Dense-only scaling - see above.
🎯 Minimum Viable Reading¶
3 hours - you need to hold a conversation about modern AI
- Transformer - the architecture
- GPT-3 - in-context learning
- InstructGPT - how it became an assistant
- Chain-of-Thought - how it reasons
+3 hours - you are building on top of models
- RAG and Dense Retrieval - grounding
- LoRA - adaptation
- ReAct and MCP - tools and agents
+3 hours - you need to reason about cost and capability
📚 Suggested Reading Orders¶
Relevance-first (recommended)¶
Transformer → GPT-3 → InstructGPT → Chain-of-Thought → RAG → LoRA → Chinchilla → MoE → DPO → Test-Time Compute → DeepSeek-R1 → RoPE → FlashAttention → vLLM → ReAct → MCP
Most useful material first; you can stop at any point and still have a coherent picture.
Chronological (how the field actually moved)¶
Word2Vec → Seq2Seq → Bahdanau Attention → ResNet → U-Net → Transformer → GPT-1 → BERT → GPT-2 → Scaling Laws → GPT-3 → ViT → DDPM → CLIP → RoPE → LoRA → Chinchilla → InstructGPT → Chain-of-Thought → Stable Diffusion → LLaMA → DPO → Mixtral → o1 → DeepSeek-R1 → GPT-5
Slower start, but the causal chain is much clearer.
Architecture-first¶
Transformer → RoPE → GQA → FlashAttention → MoE / Switch → Mamba → ViT → MAE → ResNet → U-Net → VQ-VAE → VQ-GAN → DiT
Track: image and video generation¶
VAE → GANs → DDPM → DDIM → Classifier-Free Guidance → Stable Diffusion → Imagen → DALL-E 3 → ControlNet → DreamBooth → Flow Matching / SD3 → Sora / DiT → Genie
Track: reasoning models¶
Chain-of-Thought → Self-Consistency → Tree of Thoughts → STaR → Process Reward Models → Quiet-STaR → Test-Time Compute → o1 → GRPO → RLVR → DeepSeek-R1 → rStar-Math → Meta-CoT
Track: production systems¶
RAG → Dense Retrieval → GraphRAG → LoRA → QLoRA → GPTQ & AWQ → GQA → FlashAttention → PagedAttention → Speculative Decoding → ReAct → MCP → SWE-bench → LLM-as-a-Judge → Llama Guard
💡 Pro Tips¶
Skimming a summary: read "Why This Matters", then the results table, then "Key Takeaways". Skip the derivations on a first pass.
Going deep: read the summary, then the original paper, then implement the core idea on a toy problem. The gap between "I followed that" and "I can build it" is where the learning is.
Building something: go straight to the practical-applications and limitations sections. The limitations are the more valuable half - they tell you what will break in production.
✅ Progress Tracker¶
Essentials (15)
- Transformer · [ ] Scaling Laws · [ ] Chinchilla · [ ] GPT-3 · [ ] InstructGPT
- Chain-of-Thought · [ ] RAG · [ ] LoRA · [ ] DPO · [ ] Mixtral / MoE
- RoPE · [ ] FlashAttention · [ ] Test-Time Compute · [ ] GRPO · [ ] DeepSeek-R1
Production layer (10)
- Dense Retrieval · [ ] GraphRAG · [ ] QLoRA · [ ] GPTQ & AWQ · [ ] GQA
- PagedAttention · [ ] Speculative Decoding · [ ] ReAct · [ ] MCP · [ ] SWE-bench
Historical context (9)
- Word2Vec · [ ] Seq2Seq · [ ] Bahdanau Attention · [ ] GPT-1 · [ ] GPT-2
- BERT · [ ] GANs · [ ] PPO · [ ] PaLM
Then pick a track above, or work through INDEX.md for all 107.
Last updated: 2026-08-20 · Collection: 107 papers · See also: BROWSE.md · QUICK_REFERENCE.md · ROADMAP.md