RL, alignment, and reasoning models — starting with reasoning that needs no training.
Reasoning without training
01Reasoning that needs no extra training.
Voting, best-of-N, process rewards, budget forcing.
RL foundations
03Actions, rewards, and a one-armed lesson.
States, values, and the backup equation.
DP, Monte Carlo, TD, and Q-learning.
Push up the actions that paid off.
A clipped update, with TRPO and GAE as deep dives.
Learning from preferences
08Preferences, a reward model, PPO with a KL leash.
Alignment without a reward model or an RL loop.
Training reasoners
10Group-relative advantages and a checker as the reward.
A case study of a model trained to think.
A small, runnable capstone.
Inspired by the work of Vizuara, Mohit Iyyer, Umar Jamil.