Language Models
How machines learned to read, write, and reason with human language at scale.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Ask a big model to show its work — and it starts to reason
Big language models were surprisingly bad at multi-step problems — grade-school math, logic puzzles — often blurting a wrong final answer. This paper found a shockingly simple fix that requires no training at all: in the few examples you show the model (the “prompt”), don't just show it question-and-answer pairs — show it the *reasoning in between*. Write out the step-by-step thinking that leads to each answer. The model then imitates that pattern on new problems, producing its own “chain of thought” before answering — and accuracy leaps. The headline result: prompting a 540-billion-parameter model with just eight worked examples hit 56.9% on the GSM8K math benchmark — up from 17.9% with ordinary prompting — beating the previous best (a model specially fine-tuned for the task and paired with a separate verifier, at 55%). The twist: this only works in *large* models. In small ones, showing the steps does little or even hurts. Reasoning-by-prompting is an ability that *emerges* with scale — a discovery that reshaped how people use LLMs.
QLoRA: Efficient Finetuning of Quantized LLMs
Fine-tune a 65-billion-parameter model on one gaming GPU
LoRA already made fine-tuning cheaper by training tiny add-on matrices instead of the whole model. But there was still a wall: you had to fit the giant frozen base model in GPU memory just to run it — and a 65-billion-parameter model needs over 780 GB, far beyond any single GPU. QLoRA smashes that wall by squashing the frozen base model down to 4 bits per number (instead of the usual 16), roughly a 4× shrink, so the whole thing fits in under 48 GB — a single high-end GPU. It does this without hurting quality, using three clever tricks: a new 4-bit number format (NF4) tuned to how weights are actually distributed, “double quantization” that even compresses the compression bookkeeping, and “paged optimizers” that spill memory to the CPU during brief spikes instead of crashing. The tiny LoRA add-ons still train in full precision on top. To prove it works, they fine-tuned a chatbot called Guanaco in 24 hours on one GPU — and it reached 99.3% of ChatGPT's score on a popular benchmark. QLoRA is why hobbyists and small labs can now fine-tune enormous models at home.
Retrieval-Augmented Generation for Large Language Models: A Survey
RAG — giving a language model an open book to read from
A language model only knows what it saw during training. So it can confidently make things up (“hallucinate”), it can't know anything recent, and it can't see your private documents. Retrieval-Augmented Generation (RAG) fixes this with a simple, powerful idea: before the model answers, go fetch relevant text from an external library and hand it to the model along with the question — like turning a closed-book exam into an open-book one. The basic recipe has three steps: (1) Indexing — chop your documents into chunks and store them so they're searchable by meaning; (2) Retrieval — when a question comes in, pull the few most relevant chunks; (3) Generation — feed those chunks plus the question to the model so its answer is grounded in real sources. This survey organizes the whole field into three generations of increasingly sophisticated RAG — Naive, Advanced, and Modular — and breaks every system into three parts: what you retrieve, how you use it, and how the model generates. RAG is now one of the most widely used techniques in real-world AI, because it makes answers more accurate, up-to-date, and traceable to a source.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
The paper that introduced RAG — a model with two memories
A big language model stores everything it knows inside its own weights — call that its “memory from studying.” That memory is fuzzy: the model can't quote a source, can't easily be corrected, and forgets nothing gracefully. This 2020 paper — the one that actually coined “RAG” — gives a model a *second*, separate memory: a searchable index of all of Wikipedia (about 21 million passages). When you ask a question, a **retriever** finds the most relevant passages, and a **generator** (a seq2seq model, BART) writes the answer using both its own internal knowledge *and* those retrieved passages. The whole thing is trained end-to-end. They test two flavors — one that uses a single retrieved passage for the whole answer (RAG-Sequence), and one that can draw on a different passage for each word (RAG-Token) — and set new records on open-domain question answering, while producing answers that are more specific, more factual, and *traceable to a source*. Best of all, you can update what the model knows just by swapping the index — no retraining.
Introducing the Model Context Protocol (MCP)
One universal way to plug AI into the outside world
A language model on its own is a brain in a box: brilliant at talking, but it can't see your files, query your database, or press any buttons in the real world. To fix that, people used to hand-build a custom connector for every pairing of AI-app-and-tool — a combinatorial mess that never scaled. The Model Context Protocol (MCP), open-sourced by Anthropic in November 2024, replaces all those one-off bridges with a single open standard. The official analogy: MCP is “a USB-C port for AI applications.” Just as USB-C is one socket that any device can plug into, MCP is one standard so any compliant AI app can plug into any compliant tool or data source — build it once, and it works everywhere. Under the hood it's refreshingly simple: small programs called MCP servers expose three things — Tools (actions the AI can do), Resources (data it can read), and Prompts (templates the user can pick) — and AI apps (the hosts) talk to them by trading little JSON-RPC messages. This paper walks a total beginner all the way from “why does this exist?” to writing a working MCP server in Python or TypeScript, watching a real tool call travel end-to-end, and poking a live JSON-RPC console — then hands you an “MCP Certified” badge.
Attention Is All You Need
The Transformer — the architecture behind modern AI
Before this paper, the best translation systems read sentences one word at a time, carrying a running memory forward (recurrent networks). That was slow — you can't process word 5 until word 4 is done — and forgetful over long sentences. The 2017 Transformer threw out that sequential machinery entirely and replaced it with attention: a mechanism that lets every word look directly at every other word and decide which ones matter. Because nothing waits its turn, the model trains fully in parallel on GPUs, captures long-range relationships in a single step, and set new translation records while training in a fraction of the time. This design became the foundation of GPT, BERT, Claude, and essentially all modern AI.
LoopCoder-v2
Only loop once for efficient test-time computation scaling
Looped transformers make a model “think longer” by running the same block of layers over and over — reusing the same weights to get deeper computation for free. But each extra loop normally makes inference slower and hungrier for memory. The Parallel Loop Transformer (PLT) fixes that cost, so the real question becomes: how many loops actually help? Training a family of 7B code models from scratch (on 18 trillion tokens) with 1, 2, 3, and 4 loops, the authors find a surprising, sharply non-monotonic answer: two loops is the sweet spot. One extra loop lifts SWE-bench Verified from 43.0% to 64.4% — competitive with far larger models — but a third loop makes things worse, dropping to 27.6%. Their diagnostics explain why: the second loop does the real refinement, while later loops mostly add noise, and PLT's parallelism trick charges a small fixed “tax” every loop that eventually outweighs the shrinking benefit.
BERT
Pre-training deep bidirectional transformers for language understanding
Before BERT, the strongest language models read text in one direction — left to right — so when understanding a word they could only use the words before it. BERT's insight was to read in both directions at once, letting every word draw on its full context, left and right. To train a model that way, the authors invented a clever fill-in-the-blank game: randomly hide 15% of the words and have the model guess them from everything around them (the Masked Language Model), plus a second task of judging whether one sentence naturally follows another. After this self-supervised “pre-training” on 3.3 billion words of books and Wikipedia — no human labels needed — the same model can be lightly “fine-tuned” for almost any language task by adding just one small output layer. This one recipe set new records on eleven benchmarks at once, pushing the GLUE score to 80.5% and SQuAD question-answering to a 93.2 F1.
Improving Language Understanding by Generative Pre-Training
GPT-1 — pre-train once, fine-tune for anything
In 2018, most language AI was built one task at a time: to do sentiment analysis you built a sentiment model, for question answering a question-answering model, each needing its own pile of hand-labeled examples — which are scarce and expensive. This paper (the original GPT) showed a better recipe. First, take a single Transformer and train it on a mountain of ordinary unlabeled text with one dead-simple goal: predict the next word, over and over, across 7,000+ books. That teaches it a surprising amount about grammar, facts, and reasoning — for free, no labels needed. Then, to specialize it for a real task, just keep that same model almost entirely unchanged and fine-tune it briefly on a small labeled set, feeding structured inputs (like a premise and hypothesis) as one plain sequence with a divider token. This one general model beat purpose-built, task-specific architectures on 9 of 12 benchmarks. It's the blueprint every GPT since has followed.
GPT-3
Language Models are Few-Shot Learners
People can learn a new task from just a couple of examples. If someone shows you two or three sentences and their French translations, you can usually translate the next one — no long training needed. Computers used to be bad at this. To teach a model a new task, you normally had to feed it thousands of labeled examples and retrain it. This paper asks a simple question: what if we just make the model really, really big? The authors built GPT-3, a text model with 175 billion “knobs” (parameters) — about 10 times bigger than anything like it before. Then they did something surprising: they gave it new tasks with only a few examples typed right into the prompt, and changed nothing inside the model. No retraining. It just read the examples and did the task. This is called few-shot learning, and GPT-3 turned out to be shockingly good at it — even at things like simple arithmetic and writing news articles people couldn't tell were fake.
Training Language Models to Follow Instructions with Human Feedback
InstructGPT — teaching AI to actually do what you ask
A giant language model is basically a super-powered autocomplete: it was only ever trained to guess the next word on random internet text. So when you ask it to do something, it doesn't really try to be helpful — it just continues the text in a plausible way, which can mean ignoring your request, making things up, or saying something rude. This paper fixes that with a three-step recipe. First, hire people to write good example answers and train the model to copy them. Second, have people rank several of the model's answers from best to worst, and train a second “judge” model to predict those human preferences. Third, let the main model practice, using the judge as a coach that rewards more-helpful answers. The result — called InstructGPT — follows instructions far better: people preferred the answers of a small 1.3-billion-parameter InstructGPT over the original model that was 100× bigger. It also made up facts less and was more truthful. This is the training recipe behind ChatGPT.
Learning to Summarize from Human Feedback
Teaching a model to write summaries people actually prefer
How do you train a model to write a good summary? The usual way is to have it copy human-written summaries and then score it with ROUGE — a metric that just counts overlapping words. But word-overlap isn't the same as quality; a summary can share lots of words and still be worse. This paper takes a different route: instead of optimizing a proxy, optimize for what people actually prefer. They showed humans two summaries of the same Reddit post and asked which is better — 64,832 times. From those comparisons they trained a “reward model” that predicts human preference, then used reinforcement learning to nudge a summarizer toward summaries that score high on it. The result: their 6.7-billion-parameter model's summaries were preferred over the human-written reference summaries about 65% of the time — and it beat much larger models trained the ordinary way. This is the RLHF recipe that later powered ChatGPT.
LoRA: Low-Rank Adaptation of Large Language Models
Fine-tune a giant model by training two tiny matrices
To make a big pretrained model good at your specific task, the old way was “full fine-tuning”: nudge all of its billions of weights and save a brand-new full-size copy. For a model like GPT-3, that copy is ~350 GB — and you'd need one for every task. Painful and expensive. LoRA offers a beautiful shortcut. Leave the giant model's weights completely frozen, and next to each one add a tiny detour built from two small matrices multiplied together (call it B × A). Only those small matrices get trained. Because the change a model needs to learn a task turns out to be “low-rank” (simple, not requiring a full-size adjustment), these little add-ons are enough. The savings are staggering: on GPT-3, LoRA trains about 10,000× fewer numbers, uses 3× less GPU memory, and shrinks each task's saved file from 350 GB to 35 MB — while matching or beating full fine-tuning. And once trained, you can fold the detour back into the original weights, so it runs with zero extra delay. It's how almost everyone customizes big models today.