Efficiency & Scaling
Getting more out of models for less — test-time compute, looped transformers, and the art of doing more with the same parameters.
QLoRA: Efficient Finetuning of Quantized LLMs
Fine-tune a 65-billion-parameter model on one gaming GPU
LoRA already made fine-tuning cheaper by training tiny add-on matrices instead of the whole model. But there was still a wall: you had to fit the giant frozen base model in GPU memory just to run it — and a 65-billion-parameter model needs over 780 GB, far beyond any single GPU. QLoRA smashes that wall by squashing the frozen base model down to 4 bits per number (instead of the usual 16), roughly a 4× shrink, so the whole thing fits in under 48 GB — a single high-end GPU. It does this without hurting quality, using three clever tricks: a new 4-bit number format (NF4) tuned to how weights are actually distributed, “double quantization” that even compresses the compression bookkeeping, and “paged optimizers” that spill memory to the CPU during brief spikes instead of crashing. The tiny LoRA add-ons still train in full precision on top. To prove it works, they fine-tuned a chatbot called Guanaco in 24 hours on one GPU — and it reached 99.3% of ChatGPT's score on a popular benchmark. QLoRA is why hobbyists and small labs can now fine-tune enormous models at home.
LoopCoder-v2
Only loop once for efficient test-time computation scaling
Looped transformers make a model “think longer” by running the same block of layers over and over — reusing the same weights to get deeper computation for free. But each extra loop normally makes inference slower and hungrier for memory. The Parallel Loop Transformer (PLT) fixes that cost, so the real question becomes: how many loops actually help? Training a family of 7B code models from scratch (on 18 trillion tokens) with 1, 2, 3, and 4 loops, the authors find a surprising, sharply non-monotonic answer: two loops is the sweet spot. One extra loop lifts SWE-bench Verified from 43.0% to 64.4% — competitive with far larger models — but a third loop makes things worse, dropping to 27.6%. Their diagnostics explain why: the second loop does the real refinement, while later loops mostly add noise, and PLT's parallelism trick charges a small fixed “tax” every loop that eventually outweighs the shrinking benefit.
GPT-3
Language Models are Few-Shot Learners
People can learn a new task from just a couple of examples. If someone shows you two or three sentences and their French translations, you can usually translate the next one — no long training needed. Computers used to be bad at this. To teach a model a new task, you normally had to feed it thousands of labeled examples and retrain it. This paper asks a simple question: what if we just make the model really, really big? The authors built GPT-3, a text model with 175 billion “knobs” (parameters) — about 10 times bigger than anything like it before. Then they did something surprising: they gave it new tasks with only a few examples typed right into the prompt, and changed nothing inside the model. No retraining. It just read the examples and did the task. This is called few-shot learning, and GPT-3 turned out to be shockingly good at it — even at things like simple arithmetic and writing news articles people couldn't tell were fake.
Learning to Summarize from Human Feedback
Teaching a model to write summaries people actually prefer
How do you train a model to write a good summary? The usual way is to have it copy human-written summaries and then score it with ROUGE — a metric that just counts overlapping words. But word-overlap isn't the same as quality; a summary can share lots of words and still be worse. This paper takes a different route: instead of optimizing a proxy, optimize for what people actually prefer. They showed humans two summaries of the same Reddit post and asked which is better — 64,832 times. From those comparisons they trained a “reward model” that predicts human preference, then used reinforcement learning to nudge a summarizer toward summaries that score high on it. The result: their 6.7-billion-parameter model's summaries were preferred over the human-written reference summaries about 65% of the time — and it beat much larger models trained the ordinary way. This is the RLHF recipe that later powered ChatGPT.
LoRA: Low-Rank Adaptation of Large Language Models
Fine-tune a giant model by training two tiny matrices
To make a big pretrained model good at your specific task, the old way was “full fine-tuning”: nudge all of its billions of weights and save a brand-new full-size copy. For a model like GPT-3, that copy is ~350 GB — and you'd need one for every task. Painful and expensive. LoRA offers a beautiful shortcut. Leave the giant model's weights completely frozen, and next to each one add a tiny detour built from two small matrices multiplied together (call it B × A). Only those small matrices get trained. Because the change a model needs to learn a task turns out to be “low-rank” (simple, not requiring a full-size adjustment), these little add-ons are enough. The savings are staggering: on GPT-3, LoRA trains about 10,000× fewer numbers, uses 3× less GPU memory, and shrinks each task's saved file from 350 GB to 35 MB — while matching or beating full fine-tuning. And once trained, you can fold the detour back into the original weights, so it runs with zero extra delay. It's how almost everyone customizes big models today.