List your company
ResearchFormerly WizWand

Multi-hop Reasoning on HotpotQA (EM/F1)

55F1 ScoreBGE-reranker
-1.7819212.9595427.70142.44246Nov 2, 2025Dec 7, 2025Jan 11, 2026Feb 15, 2026Mar 22, 2026Apr 26, 2026May 31, 2026
Updated 5mo ago

Evaluation results

MethodLinks
BGE-reranker2026.03554251
MergeRAG-Sym2026.035543.551.5
MergeRAG-Asym2026.03554053.5
Prompt-R1Backbone=GPT-4o-mini2025.1152.3144.53—
CoT ReasoningBackbone=GPT-4o-mini2025.1149.742.97—
RAPTOR2026.0348.934.544
GEPAOptimization Framework=APO, Backbone=GPT-4o-mini2025.1147.0338.28—
OPROOptimization Framework=APO, Backbone=GPT-4o-mini2025.1146.8334.38—
BM252026.0346.735.542.5
RECOMP2026.0344.43140
BaselineBackbone=GPT-4o-mini2025.1140.0733.59—
TextGradOptimization Framework=APO, Backbone=GPT-4o-mini2025.1137.127.34—
Tree-RAG2026.0336.227.530.8
GRPOBackbone=Qwen3-4B2025.1132.2727.34—
SFTBackbone=Qwen3-4B2025.1131.0923.44—
BaselineBackbone=Qwen3-4B2025.1124.2521.09—
CoT ReasoningBackbone=Qwen3-4B2025.1122.9818.75—
Soft-NBCEMem=O(L^2/n), zero-shot=false, distillation=true2026.050.479——
Soft-NBCEMem=O(L^2/n), zero-shot=true, distillation=false2026.050.454——
Vanilla NBCEMem=O(L^2/n)2026.050.427——
PCWMem=O(L^2/n)2026.050.405——
Truncated (8K)Mem=O(L^2)2026.050.402——
AgentsCategory=Static-Prompting2026.04——37.5
AutoPECategory=Self-Evolving2026.04——39.8
Best-of-16Strategy=Best-of-16, Inference Budget=C=162026.03——82.5
CoTCategory=Static-Prompting2026.04——38.8
CoT2-MetaStrategy=Ours (CoT2-Meta), Inference Budget=C=162026.03——90.4
DSPyCategory=Self-Evolving2026.04——43.9
Greedy CoTStrategy=Greedy CoT, Inference Budget=C=162026.03——78.5
LangMARLCategory=Ours2026.04——60.2
ReflexionCategory=Self-Evolving2026.04——59.1
ReST-MCTS*Strategy=ReST-MCTS*, Inference Budget=C=162026.03——87.4
SymbolicCategory=Self-Evolving2026.04——44.8
TextGradCategory=Self-Evolving2026.04——57.3
Vanilla ToTStrategy=Vanilla ToT, Inference Budget=C=162026.03——85.1