DeFT-RAG: Decomposing Multi-Hop Questions into Self-Contained Sub-Questions with Reusable Answers
Retrieval-augmented generation works well when one passage contains the needed answer. It is much less reliable when the answer is distributed across several documents and the system must discover hidden intermediate entities along the way. The paper by Dingyi Kang and Bingzhe Li presents DeFT-RAG as an attempt to make that process more systematic, less dependent on long prompts, and cheaper to repeat.
The problem: retrieval chains break at hidden entities
A multi-hop question may mention an album, ask about its performer, then ask about the city where that performer formed a band, and finally ask for a fact about that city. The performer and city are bridge entities: they are not stated in the original question, but later searches depend on finding them.
Standard RAG usually retrieves passages using the wording of the complete question. That makes passages about unnamed bridge entities difficult to find. Graph-based RAG can connect entities and documents, but the authors argue that methods which retrieve only once often fail to follow a chain beyond the first hidden entity. Iterative RAG improves coverage by searching step by step, yet it asks the language model to plan the chain, manage dependencies, rewrite questions, remember prior findings, and decide when to stop. Each additional responsibility creates another way to fail.
How DeFT-RAG works
DeFT-RAG divides the process into smaller, more explicit operations.
- Incremental decomposition: instead of generating a complete plan, the backbone model writes one self-contained sub-question at a time. Each question names the entities it needs and asks for the next missing fact.
- Structured memory: answers are stored as a compact table of question-and-answer pairs. This table, rather than a growing reasoning transcript, records what the system has already learned.
- Externalized decisions: a smaller fine-tuned decision model handles yes-or-no judgments, including whether passages are relevant, whether the original question can be answered directly, and whether the collected facts are sufficient.
- Answer reuse: a cache stores previous sub-questions and answers. If a later sub-question has the same meaning, DeFT-RAG can reuse the answer without another retrieval or backbone-model call.
This design does not eliminate model reasoning. The main LLM still generates sub-questions, answers them from selected passages, and produces the final answer. The intended change is to keep each generation task narrow while moving bookkeeping and classification into deterministic rules or a specialized decision model.
What the evaluation found
The authors evaluated DeFT-RAG on MuSiQue, 2WikiMultiHopQA, and HotpotQA, using 1,000 test questions from each benchmark. They compared it with 13 baselines, including vanilla RAG, seven graph-based systems, and four iterative systems. Experiments used both gpt-4o-mini and Qwen3.6-35B-A3B as the backbone models.
According to the paper, DeFT-RAG achieved the highest exact-match, token-F1, and LLM-judged accuracy in all six benchmark-and-backbone settings. Its largest reported accuracy advantage was on MuSiQue, where it scored 0.629 versus 0.577 for the strongest competing system with gpt-4o-mini, and 0.636 versus 0.583 with Qwen3.6. On 2WikiMultiHopQA and HotpotQA, the gains were smaller, and DeFT-RAG was sometimes statistically on par with the best baseline rather than clearly ahead.
The evidence-chain analysis helps explain the result. On MuSiQue, DeFT-RAG supplied the LLM with all passages needed for 46% of three-hop questions and 18% of four-hop questions. The corresponding figure for three-hop questions was 41% for Self-Ask and 28% for LinearRAG. DeFT-RAG also supplied fewer passage words to the LLM: 390 per question, compared with 870 for vanilla RAG and 2,590 for Self-Ask in the reported comparison.
Passage filtering mattered. When the authors disabled DeFT-RAG’s relevance selection and simply passed the top five retrieved passages to the LLM, accuracy fell by 2.3 points on MuSiQue, 3.7 on 2WikiMultiHopQA, and 5.3 on HotpotQA in a 300-question-per-benchmark study. Fine-tuning the decision model also improved end-to-end MuSiQue accuracy in the reported subset, from 0.587 to 0.623.
Reuse reduces calls, but not necessarily latency
The cache experiment used 1,000 MuSiQue questions processed in sequence. The authors report that 23.1% of generated sub-questions were answered from the cache, reducing LLM calls by 12.9% without a measured accuracy decrease. This is particularly relevant for enterprise collections where many user questions revisit the same people, organizations, products, or locations.
There is an important trade-off. DeFT-RAG used fewer input tokens than other iterative methods and made fewer backbone-model calls than Self-Ask and Hi-Q, but its median latency was the highest in the reported comparison: 9.9 seconds. Most of that delay came from repeated local decision-model calls. Replacing that model with a faster closed decision service reduced median latency to 5.0 seconds, while reported accuracy stayed within 1.2 points of the local-model version.
Limitations and open questions
The method remains vulnerable to errors in the backbone model. The paper’s case studies show failures when a generated sub-question is not truly self-contained, when the system asks for the wrong property of the correct entity, and when the stopping decision accepts an incomplete set of facts. Nested comparison questions in 2WikiMultiHopQA were especially difficult.
The results also come from curated benchmarks and shared passage corpora, not from changing web pages, noisy enterprise documents, or adversarial retrieval environments. Cache safety depends on correctly judging whether two questions have the same meaning; a false match could propagate an incorrect answer across many requests. In addition, the paper reports that one decision threshold was selected during development using stored answers from part of the test-question pool, then fixed for later runs. That choice is disclosed, but it makes strict interpretation of the headline comparison more complicated.
Practical implications
For multi-hop RAG systems, DeFT-RAG suggests a useful engineering principle: separate generation from control flow. Keep a compact, inspectable fact table; ask for one grounded sub-question at a time; filter passages before sending them to the expensive model; and cache only answers whose entity and property match can be verified conservatively.
This architecture is most promising when questions share intermediate facts and answer quality matters more than minimum single-query latency. Teams should benchmark it against simpler RAG on their own corpus, measure decision-model overhead, and monitor cache false positives. The paper demonstrates a strong benchmark result, but whether the same gains hold in production depends on corpus quality, question repetition, model serving speed, and the reliability of the specialized decisions.
Sources
AI-assisted research summary. Check the original paper for complete methods, results, and limitations.