FINDING NEEDLES · CHAPTER 20

Chroma Context-1

Chapter 19 wrote the search loop by hand. Context-1 is a model trained to run that loop itself: search, read, throw away what is useless, and hand back the documents that matter.

~15 min readCase studyTested Python

What you'll learn

  • What problem a retrieval subagent solves, and why its context fills up
  • The four tools Context-1 can call, and the harness rules around them
  • How its training tasks are made and checked
  • How a reward that favours recall changes which answer list wins
  • How each piece maps back to earlier chapters of this series

Before you start

The scout and the general

Recall the hut agent from chapter 19. It searched twice, read four notes, and stopped. Now picture a question that needs fifteen searches. Every result is pasted into the model’s context. By search ten, most of what it is reading is notes it already decided were useless.

That pile-up has a cost. Long inputs are slower and dearer to run, and Chroma’s own earlier study found that model performance “grows increasingly unreliable as input length grows”. They call this context rot.

Context-1 splits the work. A small model acts as a scout: it searches, reads, and keeps only the useful documents. It does not answer the question. It hands a ranked list of documents to a separate, larger model that writes the answer. And the scout can tear pages out of its own notebook: a tool that deletes chunks from its context once it has judged them useless.

The facts in one table

WhatReported
JobRetrieve supporting documents for complex, multi-hop questions. Return them ranked. Never answer.
Size and base20B parameters, fine-tuned from gpt-oss-20b
Training dataOver 8,000 synthetic search tasks
Training methodSupervised fine-tuning, then reinforcement learning with verifiable rewards
ReleasedMarch 2026, weights under Apache 2.0, data-generation code public
ServingvLLM on an Nvidia B200, MoE layers in MXFP4; 400–500 tokens per second end to end

The interesting part is not the size. It is what the model is allowed to do at each step, and what it is rewarded for. The next sections take those in turn.

scout · budget · mission · pack

MANIM Context packing: scout retrieves; general spends the window.

Four tools and a harness

The model runs inside a harness: a program that executes its tool calls against a document collection and appends the results to the conversation. At each turn the model may call any of four tools, several at once.

ToolWhat it does
search_corpus(query)Hybrid BM25 and dense search, fused with reciprocal rank fusion. Takes 50 candidates, reranks them, returns the top results within a token budget.
grep_corpus(pattern)A regular-expression search. Returns up to 5 matching chunks.
read_document(doc_id)Reads one document, its chunks reranked and cut to fit the budget that is left.
prune_chunks(chunk_ids)Removes the listed chunks from the context. The harness still keeps the full history for scoring.

Three harness rules matter as much as the tools.

  1. Never the same chunk twice. The harness tracks every chunk id any search has returned and excludes them from later searches. This is the hut agent’s “seen notes” list, enforced outside the model.
  2. The budget is always visible. Every observation ends with a usage line, for example [Token usage: 14,203/32,768].
  3. Pressure, then a wall. Past a soft threshold, the harness suggests pruning (only during training). Past a hard cutoff, every tool except prune_chunks is rejected. The model must prune or finish.

So the model’s choices narrow as the context fills. Early on it can search freely. Later it has to decide what to forget before it can look further.

Hard to solve, easy to check

Reinforcement learning needs questions with known answers and known supporting documents. Context-1 is trained on synthetic multi-hop tasks generated in four domains: web pages, SEC filings, patents, and email. Each task has clues, a question, an answer, and the documents that support it, plus distractor documents that look right and are not.

A bad label teaches the wrong thing, so every task is checked by extraction. An LLM pulls two sets of quotes: spans from each supporting document, and the clue each span supports. Then a plain program checks that each document quote really appears, word for word, in the source. A task that fails the check is dropped. The report states that human checkers agreed with this automated judge more than 80% of the time across domains.

The trick is that the expensive judgment, “does this document support this clue?”, is turned into a cheap one: “is this quote in this document?”. That is a string search, and it cannot be fooled by a confident model.

Copy a teacher, then learn from reward

Supervised fine-tuning first. Large models, such as Kimi K2.5, run the same harness and produce full search episodes. The small model learns to imitate them: valid tool calls, parallel searches, pruning.

Then reinforcement learning. Each step takes 128 queries and runs 8 rollouts of each, 1,024 episodes. Rollouts of the same query are scored against each other, so a group where all 8 earn the same reward teaches nothing and is dropped. The optimiser is CISPO. In the related method GRPO, tokens whose update ratio falls outside a clip range get no gradient at all. CISPO keeps a weighted signal for them, so rare but decisive tokens still learn. Of about 300 planned steps, training converged near 230.

The reward has several parts:

PartRuleWhy
OutcomeAn F-score on the final list. At the start recall counts 16 times as much as precision, annealed to 4 times.A scout that misses a document is worse than one that returns an extra.
Trajectory recallCredit for every gold chunk the agent ever saw, even if pruned.Rewards exploring, not one broad search and an early stop.
Final answer bonus+1.0 if a returned chunk contains the answerOn-topic is not enough.
Prune penalty0.1 per prune call past a streak of 3, capped at 0.5Prune in batches, not one chunk per turn.
Turn penalty0 at 64 turns, rising in a line to 0.5 at 128Stop wandering.

“Recall counts 16 times as much” is an F-score with β² = 16, so β = 4. Annealing to 4 times is β = 2. To see what that does, take one question with 5 gold chunks and two possible final lists.

a weighted harmonic mean of precision P and recall R. β = 1 weighs them equally. A larger β makes recall count more.

Final listPRF1F2 (late)F4 (early)
Wide: 10 chunks, 4 gold0.40.80.5330.6670.756
Tight: 4 chunks, 3 gold0.750.60.6670.6250.607

Under F1 the tight list wins. Under the training reward the wide list wins, by more at the start (0.756 against 0.607) than at the end (0.667 against 0.625). That is the scout’s job written as arithmetic: find everything first, then learn to be tidier. The penalties are small next to that. Running 96 turns costs 0.25. A streak of 5 single-chunk prunes costs 0.2.

What training changed

The report scores a returned list four ways. Trajectory recall: the share of gold documents the agent saw at any point. Output recall: the share in its final list. F1 on the final list. Final answer found: whether any returned chunk contains the answer. That last one deliberately skips the answering model, so a failure can be blamed on the search and not on the writer.

Metricgpt-oss-20bContext-1Change
Trajectory recall0.6400.739+0.099
Output recall0.3610.641+0.280
F10.3070.487+0.180
Final answer found0.5410.798+0.257

Look at where the gain is. Trajectory recall moved by +0.099: the trained model sees a bit more. Output recall moved by +0.280: it keeps far more of what it sees. Divide one by the other. The base model handed back 0.361 / 0.640 = 0.564 of the gold it had found. Context-1 hands back 0.641 / 0.739 = 0.867. Most of what training taught was judgment, not search.

The behaviour changed with it. Context-1 makes 2.56 tool calls per turn against 1.52, finishes in 5.2 turns against 6.7, and its prune decisions are accurate 0.941 of the time against 0.824. On the public BrowseComp-Plus benchmark, its final-answer-found rate is 0.87 for one run, and 0.96 when four runs are fused with reciprocal rank fusion.

Every piece, you have met before

In Context-1In this series
BM25 plus dense, fused with RRF10 · Hybrid search
50 candidates, reranked09 · Reranking
Grading a retrieval, asking again on failure14 · Beyond the basic pipeline
Trajectory recall, output recall, F-scores15 · Evaluating search and RAG
Questions that span several documents16 · Graphs and structured retrieval
A fixed budget for every request18 · Production search architecture
The loop, seen-chunk exclusion, stopping19 · Agentic search
Four runs fused with RRF10 · Hybrid search, applied to whole agent runs

What is new is prune_chunks as an action the policy learns, and a reward written for a scout rather than an answerer.

The numbers you can re-run

The reward rules and the results are the report’s. The two example lists are ours.

context1_numbers.pyF4 0.756 vs 0.607 · kept 0.867
"""The arithmetic behind the Context-1 case study.

The reward rules and the reported results are from Chroma's Context-1
report (March 2026). The two example episodes are ours, chosen to make
the recall weighting visible.
"""


def f_beta(precision, recall, beta):
    """F-score where recall counts beta squared times as much as precision."""
    b2 = beta * beta
    return (1 + b2) * precision * recall / (b2 * precision + recall)


def episode(returned, gold, hits):
    """Precision and output recall for one finished search."""
    return hits / returned, hits / gold


# Same question, 5 gold chunks. One agent hands back a wide list, one a tight list.
WIDE = episode(returned=10, gold=5, hits=4)     # precision 0.4, recall 0.8
NARROW = episode(returned=4, gold=5, hits=3)    # precision 0.75, recall 0.6


def turn_penalty(turns, start=64, end=128, cap=0.5):
    """0 up to 64 turns, rising in a straight line to 0.5 at 128."""
    if turns <= start:
        return 0.0
    return min(cap, cap * (turns - start) / (end - start))


def prune_penalty(streak, free=3, per_call=0.1, cap=0.5):
    """0.1 for each prune call past a streak of 3 in a row, at most 0.5."""
    return round(min(cap, per_call * max(0, streak - free)), 3)


# One RL step: 128 queries, 8 rollouts each.
QUERIES_PER_STEP = 128
ROLLOUTS_PER_QUERY = 8

# Reported results: the base model (gpt-oss-20b) versus Context-1.
BASE = {"trajectory recall": 0.640, "output recall": 0.361,
        "F1": 0.307, "final answer found": 0.541}
CONTEXT1 = {"trajectory recall": 0.739, "output recall": 0.641,
            "F1": 0.487, "final answer found": 0.798}


def gains():
    return {k: round(CONTEXT1[k] - BASE[k], 3) for k in BASE}


def kept_share(scores):
    """Of the gold chunks the agent saw, what share made it to the output."""
    return round(scores["output recall"] / scores["trajectory recall"], 3)

Mistakes learners make

Reading Context-1 as a question-answering model. It returns ranked documents. A separate model writes the answer. Its scores measure the search, not the answer.

Treating pruning as forgetting. Pruned chunks leave the model’s context, not the record. The harness keeps the full history, which is how trajectory recall can credit a chunk that was later pruned.

Reading a recall-heavy reward as “precision does not matter”. It still counts, and its weight grows during training, from 16 times to 4 times behind recall.

Comparing 0.96 to a single run of another model. The 4× figure fuses four Context-1 runs. Compare it to other fused systems, or use the one-run 0.87.

What to remember

  • Context-1 is a 20B search subagent from gpt-oss-20b. It returns ranked documents and never answers.
  • Four tools: search_corpus, grep_corpus, read_document, prune_chunks. The harness excludes seen chunks, shows the budget, and blocks everything but pruning when it is full.
  • Training tasks are verified by checking that extracted quotes appear word for word in their source.
  • The reward weighs recall 16 times, then 4 times, over precision. In our example that makes the wide list win, 0.756 against 0.607 at the start.
  • Output recall rose from 0.361 to 0.641. The share of seen gold it keeps rose from 0.564 to 0.867.

Check yourself

1. Why does the harness exclude chunks that were already returned?

So every search brings something new. Without it, an agent that rephrases slightly gets the same chunks again and burns budget.

2. Why reward trajectory recall when only the final list goes to the answerer?

Without it, the policy learns that one broad search and an early stop is safe. Credit for what it saw pushes it to explore.

3. Under F1 the tight list scores 0.667 and the wide one 0.533. Why does the training reward prefer the wide one?

With β = 4, recall counts 16 times as much. The wide list has recall 0.8 against 0.6, and that outweighs its lower precision: 0.756 against 0.607.

4. Trajectory recall improved by +0.099 and output recall by +0.280. What did training mostly teach?

Keeping the right documents. The trained model passes on 0.867 of the gold it sees, against 0.564 before.

Go to the source

Keep going