Why the new generation of “reasoning LLMs” is so exciting, and how the authors trained their model to actually think. (This is a translation of my original post in Russian).

Overview

In recent months, there’s been a growing buzz around “reasoning models,” i.e., large language models (LLMs) that can do more than just continue a text in a plausible style: they solve complex tasks step by step, using an internal chain-of-thought (CoT). The first notable demonstrations of such a technique emerged with OpenAI’s “o1” (though many details remain under wraps). Recently, the DeepSeek team stirred the community by releasing two open-source variants, R1 and R1-Zero, built on top of their own large Mixture-of-Experts (MoE) model, DeepSeek-V3.

I won’t dive into the “whose model is better—o1 or R1?” debate here. Instead, I’ll focus on the main technical highlights of R1, why the Zero version is especially exciting, and how the authors managed to teach the model to reason effectively.

*Comparison of metrics for DeepSeek vs. OpenAI]*

Why is everyone talking about “reasoning” again?

Some folks ask:

“What’s really new here? We’ve known about chain-of-thought prompting for a while; it works decently. Models have long been able to ‘think out loud’ and solve tasks step-by-step.”

True, the idea that LLMs can reason internally isn’t new. It’s been known for some time that if you prompt a language model to “think carefully before answering,” it can increase accuracy. After all, even humans often do math “by the column,” or break down logic step-by-step.

But what’s the catch? Previous models—like GPT-4—hit a limit if you make them think too long. Their performance can decrease due to hallucinations, context window issues, and the infamous “lost in the middle” problem (where content in the middle of a long prompt gets forgotten).

In contrast, the latest generation (o1, R1, etc.) doesn’t show that same limitation: the benefit from chain-of-thought reasoning keeps increasing the longer they “think.” The folks at OpenAI call this phenomenon “test-time scaling” — you can keep throwing more inference compute at these models at runtime (not just more during pretraining), and they keep improving.

What did the DeepSeek team release?

DeepSeek provided eight models:

DeepSeek-R1
DeepSeek-R1-Zero
Plus six distilled versions onto smaller backbones (Qwen 1.5B, 7B, 14B, 32B, Llama

...

Posts

Wikitag Contributions

Comments