FINDING NEEDLES · CHAPTER 20
Chapter 19 wrote the search loop by hand. Context-1 is a model trained to run that loop itself: search, read, throw away what is useless, and hand back the documents that matter.
Recall the hut agent from chapter 19. It searched twice, read four notes, and stopped. Now picture a question that needs fifteen searches. Every result is pasted into the model’s context. By search ten, most of what it is reading is notes it already decided were useless.
That pile-up has a cost. Long inputs are slower and dearer to run, and Chroma’s own earlier study found that model performance “grows increasingly unreliable as input length grows”. They call this context rot.
Context-1 splits the work. A small model acts as a scout: it searches, reads, and keeps only the useful documents. It does not answer the question. It hands a ranked list of documents to a separate, larger model that writes the answer. And the scout can tear pages out of its own notebook: a tool that deletes chunks from its context once it has judged them useless.
| What | Reported |
|---|---|
| Job | Retrieve supporting documents for complex, multi-hop questions. Return them ranked. Never answer. |
| Size and base | 20B parameters, fine-tuned from gpt-oss-20b |
| Training data | Over 8,000 synthetic search tasks |
| Training method | Supervised fine-tuning, then reinforcement learning with verifiable rewards |
| Released | March 2026, weights under Apache 2.0, data-generation code public |
| Serving | vLLM on an Nvidia B200, MoE layers in MXFP4; 400–500 tokens per second end to end |
The interesting part is not the size. It is what the model is allowed to do at each step, and what it is rewarded for. The next sections take those in turn.
scout · budget · mission · pack
The model runs inside a harness: a program that executes its tool calls against a document collection and appends the results to the conversation. At each turn the model may call any of four tools, several at once.
| Tool | What it does |
|---|---|
search_corpus(query) | Hybrid BM25 and dense search, fused with reciprocal rank fusion. Takes 50 candidates, reranks them, returns the top results within a token budget. |
grep_corpus(pattern) | A regular-expression search. Returns up to 5 matching chunks. |
read_document(doc_id) | Reads one document, its chunks reranked and cut to fit the budget that is left. |
prune_chunks(chunk_ids) | Removes the listed chunks from the context. The harness still keeps the full history for scoring. |
Three harness rules matter as much as the tools.
[Token usage: 14,203/32,768].prune_chunks is rejected. The model must prune or finish.So the model’s choices narrow as the context fills. Early on it can search freely. Later it has to decide what to forget before it can look further.
Reinforcement learning needs questions with known answers and known supporting documents. Context-1 is trained on synthetic multi-hop tasks generated in four domains: web pages, SEC filings, patents, and email. Each task has clues, a question, an answer, and the documents that support it, plus distractor documents that look right and are not.
A bad label teaches the wrong thing, so every task is checked by extraction. An LLM pulls two sets of quotes: spans from each supporting document, and the clue each span supports. Then a plain program checks that each document quote really appears, word for word, in the source. A task that fails the check is dropped. The report states that human checkers agreed with this automated judge more than 80% of the time across domains.
The trick is that the expensive judgment, “does this document support this clue?”, is turned into a cheap one: “is this quote in this document?”. That is a string search, and it cannot be fooled by a confident model.
Supervised fine-tuning first. Large models, such as Kimi K2.5, run the same harness and produce full search episodes. The small model learns to imitate them: valid tool calls, parallel searches, pruning.
Then reinforcement learning. Each step takes 128 queries and runs 8 rollouts of each, 1,024 episodes. Rollouts of the same query are scored against each other, so a group where all 8 earn the same reward teaches nothing and is dropped. The optimiser is CISPO. In the related method GRPO, tokens whose update ratio falls outside a clip range get no gradient at all. CISPO keeps a weighted signal for them, so rare but decisive tokens still learn. Of about 300 planned steps, training converged near 230.
The reward has several parts:
| Part | Rule | Why |
|---|---|---|
| Outcome | An F-score on the final list. At the start recall counts 16 times as much as precision, annealed to 4 times. | A scout that misses a document is worse than one that returns an extra. |
| Trajectory recall | Credit for every gold chunk the agent ever saw, even if pruned. | Rewards exploring, not one broad search and an early stop. |
| Final answer bonus | +1.0 if a returned chunk contains the answer | On-topic is not enough. |
| Prune penalty | 0.1 per prune call past a streak of 3, capped at 0.5 | Prune in batches, not one chunk per turn. |
| Turn penalty | 0 at 64 turns, rising in a line to 0.5 at 128 | Stop wandering. |
“Recall counts 16 times as much” is an F-score with β² = 16, so β = 4. Annealing to 4 times is β = 2. To see what that does, take one question with 5 gold chunks and two possible final lists.
a weighted harmonic mean of precision P and recall R. β = 1 weighs them equally. A larger β makes recall count more.
| Final list | P | R | F1 | F2 (late) | F4 (early) |
|---|---|---|---|---|---|
| Wide: 10 chunks, 4 gold | 0.4 | 0.8 | 0.533 | 0.667 | 0.756 |
| Tight: 4 chunks, 3 gold | 0.75 | 0.6 | 0.667 | 0.625 | 0.607 |
Under F1 the tight list wins. Under the training reward the wide list wins, by more at the start (0.756 against 0.607) than at the end (0.667 against 0.625). That is the scout’s job written as arithmetic: find everything first, then learn to be tidier. The penalties are small next to that. Running 96 turns costs 0.25. A streak of 5 single-chunk prunes costs 0.2.
The report scores a returned list four ways. Trajectory recall: the share of gold documents the agent saw at any point. Output recall: the share in its final list. F1 on the final list. Final answer found: whether any returned chunk contains the answer. That last one deliberately skips the answering model, so a failure can be blamed on the search and not on the writer.
| Metric | gpt-oss-20b | Context-1 | Change |
|---|---|---|---|
| Trajectory recall | 0.640 | 0.739 | +0.099 |
| Output recall | 0.361 | 0.641 | +0.280 |
| F1 | 0.307 | 0.487 | +0.180 |
| Final answer found | 0.541 | 0.798 | +0.257 |
Look at where the gain is. Trajectory recall moved by +0.099: the trained model sees a bit more. Output recall moved by +0.280: it keeps far more of what it sees. Divide one by the other. The base model handed back 0.361 / 0.640 = 0.564 of the gold it had found. Context-1 hands back 0.641 / 0.739 = 0.867. Most of what training taught was judgment, not search.
The behaviour changed with it. Context-1 makes 2.56 tool calls per turn against 1.52, finishes in 5.2 turns against 6.7, and its prune decisions are accurate 0.941 of the time against 0.824. On the public BrowseComp-Plus benchmark, its final-answer-found rate is 0.87 for one run, and 0.96 when four runs are fused with reciprocal rank fusion.
| In Context-1 | In this series |
|---|---|
| BM25 plus dense, fused with RRF | 10 · Hybrid search |
| 50 candidates, reranked | 09 · Reranking |
| Grading a retrieval, asking again on failure | 14 · Beyond the basic pipeline |
| Trajectory recall, output recall, F-scores | 15 · Evaluating search and RAG |
| Questions that span several documents | 16 · Graphs and structured retrieval |
| A fixed budget for every request | 18 · Production search architecture |
| The loop, seen-chunk exclusion, stopping | 19 · Agentic search |
| Four runs fused with RRF | 10 · Hybrid search, applied to whole agent runs |
What is new is prune_chunks as an action the policy learns, and a reward written for a scout rather than an answerer.
The reward rules and the results are the report’s. The two example lists are ours.
"""The arithmetic behind the Context-1 case study.
The reward rules and the reported results are from Chroma's Context-1
report (March 2026). The two example episodes are ours, chosen to make
the recall weighting visible.
"""
def f_beta(precision, recall, beta):
"""F-score where recall counts beta squared times as much as precision."""
b2 = beta * beta
return (1 + b2) * precision * recall / (b2 * precision + recall)
def episode(returned, gold, hits):
"""Precision and output recall for one finished search."""
return hits / returned, hits / gold
# Same question, 5 gold chunks. One agent hands back a wide list, one a tight list.
WIDE = episode(returned=10, gold=5, hits=4) # precision 0.4, recall 0.8
NARROW = episode(returned=4, gold=5, hits=3) # precision 0.75, recall 0.6
def turn_penalty(turns, start=64, end=128, cap=0.5):
"""0 up to 64 turns, rising in a straight line to 0.5 at 128."""
if turns <= start:
return 0.0
return min(cap, cap * (turns - start) / (end - start))
def prune_penalty(streak, free=3, per_call=0.1, cap=0.5):
"""0.1 for each prune call past a streak of 3 in a row, at most 0.5."""
return round(min(cap, per_call * max(0, streak - free)), 3)
# One RL step: 128 queries, 8 rollouts each.
QUERIES_PER_STEP = 128
ROLLOUTS_PER_QUERY = 8
# Reported results: the base model (gpt-oss-20b) versus Context-1.
BASE = {"trajectory recall": 0.640, "output recall": 0.361,
"F1": 0.307, "final answer found": 0.541}
CONTEXT1 = {"trajectory recall": 0.739, "output recall": 0.641,
"F1": 0.487, "final answer found": 0.798}
def gains():
return {k: round(CONTEXT1[k] - BASE[k], 3) for k in BASE}
def kept_share(scores):
"""Of the gold chunks the agent saw, what share made it to the output."""
return round(scores["output recall"] / scores["trajectory recall"], 3)
Reading Context-1 as a question-answering model. It returns ranked documents. A separate model writes the answer. Its scores measure the search, not the answer.
Treating pruning as forgetting. Pruned chunks leave the model’s context, not the record. The harness keeps the full history, which is how trajectory recall can credit a chunk that was later pruned.
Reading a recall-heavy reward as “precision does not matter”. It still counts, and its weight grows during training, from 16 times to 4 times behind recall.
Comparing 0.96 to a single run of another model. The 4× figure fuses four Context-1 runs. Compare it to other fused systems, or use the one-run 0.87.
So every search brings something new. Without it, an agent that rephrases slightly gets the same chunks again and burns budget.
Without it, the policy learns that one broad search and an early stop is safe. Credit for what it saw pushes it to explore.
With β = 4, recall counts 16 times as much. The wide list has recall 0.8 against 0.6, and that outweighs its lower precision: 0.756 against 0.607.
Keeping the right documents. The trained model passes on 0.867 of the gold it sees, against 0.564 before.