DecisionRank: On-Policy Distillation for Decision-Model Rerankers
Large language models can rerank search results effectively, but the most capable listwise systems often generate a ranking one document identifier at a time. That approach makes inference expensive: a system must decode a sequence, sometimes across a long list of candidates. DecisionRank, presented by authors from Meta and Purdue University, explores a different compromise: transfer a generative model’s ranking knowledge into a decision model that scores every candidate in one forward pass.
The problem: sequential knowledge, single-pass inference
A generative teacher expresses its preference as a sequence. After selecting one document, it evaluates the remaining documents and chooses the next. A decision-model student, by contrast, produces a vector of scores at its first output position. Sorting those first-token document-ID scores yields the final ranking, but the model does not naturally expose the same conditional decisions as the teacher.
Static distillation can provide an offline ranking target, yet it does not respond to the student’s changing predictions. The central question for DecisionRank is therefore how to use the teacher’s sequential feedback without training the student to generate a sequence that it will not generate at inference time.
Item-space on-policy distillation
The proposed method uses what the authors call item-space on-policy distillation. The student first ranks candidates using its current first-token scores. The researchers take the student’s top candidates and treat that ordering as a sequence of selections. A frozen generative teacher then evaluates those selections with teacher forcing, producing conditional choice distributions over the remaining candidate documents.
The student’s scores are normalized over the same remaining documents, and the two distributions are aligned by candidate identity rather than by vocabulary token. This matters because the teacher and student can use different document-ID representations and need not share an architecture. A reverse Kullback-Leibler objective encourages the student’s score distribution to match the teacher’s conditional preferences at the states visited by the student.
DecisionRank combines this on-policy signal with offline RankNet supervision. The offline loss supplies global ordering information from teacher-generated rankings, while the on-policy loss refines the scores around the student’s current top-ranked choices. At inference, none of the teacher’s sequence generation is required: the student processes the query and candidate set once, then sorts its first-token scores.
What the experiments show
The authors evaluate Qwen3-based students with 1.7B, 4B, and 8B parameters. The main teacher is Qwen3.8-27B, with additional experiments using Gemma-4-31B and ReasonRank-32B. Training uses 10,000 MS MARCO queries and 6,000 synthetic reasoning-oriented queries. Evaluation covers TREC Deep Learning, seven BEIR datasets, BRIGHT, and R2MED, using 100 retrieved candidates and nDCG@10 as the primary metric.
The reported results are strong across the four benchmark groups. The 8B DecisionRank student reaches 64.34 on TREC DL, 57.98 on BEIR, 36.86 on BRIGHT, and 43.55 on R2MED, with scores reported as nDCG@10 multiplied by 100. Its largest advantages are on the general retrieval benchmarks: the authors report improvements of 1.42 points over listwise DirectRank on TREC DL and 2.20 points on BEIR. On BRIGHT, the 8B model scores higher than DirectRank, while on R2MED it remains below that comparison by the reported benchmark average.
Model size is not the only notable result. The 1.7B student outperforms the compared baselines on the TREC DL and BEIR benchmark averages in the paper’s table. The authors also report that the 8B student is 24.5 times faster than listwise DirectRank on TREC DL and 16.7 times faster on BRIGHT under their stated evaluation setup. All three students formed the local effectiveness-latency Pareto frontier in those experiments.
Why the training objective matters
Ablations suggest that the alignment between training and inference is important. Item-space OPD was the only added objective that consistently improved RankNet across TREC DL, BEIR, BRIGHT, and R2MED. Offline KL, autoregressive supervised fine-tuning, autoregressive OPD, and listwise GRPO each improved some benchmark averages but reduced others.
Combining offline RankNet with item-space OPD produced the best student result in the component ablation. Across three seeds, joint training improved over RankNet for both 4B and 8B students on TREC DL and BRIGHT. The authors interpret this as complementary supervision: offline rankings stabilize the global order, while on-policy feedback corrects the student’s current decisions.
The paper also reports lower pairwise calibration error for DecisionRank students than for their corresponding generative teachers on TREC DL and BRIGHT. This improvement was obtained without a separate calibration objective. It is evidence that the distilled score differences can be useful not only for ordering documents but also for expressing confidence in pairwise preferences.
Practical implications and limitations
For search and retrieval engineers, the practical lesson is straightforward: a large generative reranker does not necessarily need to remain in the production path. Its ranking behavior can supervise a smaller decision model whose inference cost is closer to a conventional scoring pass. This is especially relevant when reranking 100 candidates, where autoregressive decoding can become a noticeable service bottleneck.
Still, the evidence should be read within the paper’s scope. The evaluations use particular first-stage retrievers, candidate sets of up to 100 documents, passage-length limits, model families, and latency configurations. The reported speedups are therefore not universal guarantees; hardware, batching, context length, implementation, and baseline configuration will affect real-world results. The training data is also limited to the datasets and synthetic queries described by the authors, so broader domain transfer remains an empirical question despite the positive R2MED and cross-teacher results.
DecisionRank demonstrates a compelling design pattern: distill a sequential teacher in the space of decisions that the student actually makes. The method does not eliminate the need for a strong generative teacher, but it shifts that cost to training and preserves single-pass ranking at deployment. For applications where relevance quality and latency must coexist, that separation could be more valuable than simply scaling the generative reranker.
Sources
AI-assisted research summary. Check the original paper for complete methods, results, and limitations.