Question Answering on HotpotQA 296:204 (test)
67.23Answerable EMLlama3-8B-Instruct
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Llama3-8B-InstructTraining Paradigm=SFT based, Backbone=Llama3-8B-Instruct, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 67.23 | 86.38 | 87.5 | 49.51 | 50.49 | 72.4 | 69 | |
| ARENA-7BTraining Paradigm=RL based, Backbone=ARENA-7B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 65.54 | 86.27 | 88.18 | 5.88 | 9.31 | 55.6 | 48.75 | |
| GRACE-Qwen3-4BTraining Paradigm=Our methods, Backbone=Qwen3-4B, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 65.2 | 84.95 | 91.55 | 66.18 | 66.18 | 81.2 | 78.87 | |
| GRACE-Llama3.1-8BTraining Paradigm=Our methods, Backbone=Llama3.1-8B, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 65.2 | 86.4 | 88.85 | 42.16 | 44.61 | 70.8 | 66.73 | |
| ARENA on Qwen3-4BTraining Paradigm=RL based, Method Strategy=ARENA, Backbone=Qwen3-4B, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 64.86 | 86.05 | 90.54 | 38.73 | 38.73 | 69.4 | 64.64 | |
| Qwen3-4BTraining Paradigm=SFT based, Backbone=Qwen3-4B, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 64.53 | 84.77 | 84.12 | 44.12 | 44.12 | 67.8 | 64.12 | |
| ARENA-8BTraining Paradigm=RL based, Backbone=ARENA-8B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 64.53 | 85.43 | 87.84 | 0 | 0 | 52 | 43.92 | |
| Search-R1-7BTraining Paradigm=Agentic Models, Backbone=Search-R1-7B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 56.08 | 79.49 | 78.72 | 47.55 | 47.55 | 66 | 63.13 | |
| SuRe on DeepSeek-chatTraining Paradigm=Prompt based, Method Strategy=SuRe, Backbone=DeepSeek-chat, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 51.69 | 78.49 | 84.12 | 0 | 0.98 | 50.2 | 42.55 | |
| R1-Searcher-7BTraining Paradigm=Agentic Models, Backbone=R1-Searcher-7B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 51.35 | 74.59 | 69.59 | 65.2 | 65.2 | 67.8 | 67.4 | |
| SimpleDeepSearcher-7BTraining Paradigm=Agentic Models, Backbone=SimpleDeepSearcher-7B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 45.61 | 68.47 | 63.18 | 64.71 | 64.71 | 63.8 | 63.94 | |
| SuRe on Llama3-8B-InstructTraining Paradigm=Prompt based, Method Strategy=SuRe, Backbone=Llama3-8B-Instruct, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 35.14 | 69.18 | 83.11 | 0 | 0 | 49.2 | 41.55 | |
| DeepSeek-chatTraining Paradigm=Prompt based, Backbone=DeepSeek-chat, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 33.45 | 60.46 | 68.92 | 75 | 75.98 | 71.8 | 72.45 | |
| SuRe on Qwen3-4BTraining Paradigm=Prompt based, Method Strategy=SuRe, Backbone=Qwen3-4B, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 26.35 | 63.62 | 65.54 | 0 | 1.47 | 39.4 | 33.51 | |
| Qwen3-4BTraining Paradigm=Prompt based, Backbone=Qwen3-4B, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 17.57 | 54.31 | 89.53 | 16.18 | 60.78 | 77.8 | 75.16 | |
| Llama3-8B-InstructTraining Paradigm=Prompt based, Backbone=Llama3-8B-Instruct, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 8.78 | 39.15 | 27.36 | 48.53 | 89.22 | 52.6 | 58.29 | |
| TrustAlign-3BTraining Paradigm=RL based, Backbone=TrustAlign-3B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 0 | 38.66 | 44.59 | 55.88 | 67.16 | 53.8 | 55.88 | |
| TrustAlign on Qwen3-4BTraining Paradigm=RL based, Method Strategy=TrustAlign, Backbone=Qwen3-4B, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 0 | 32.27 | 11.49 | 0 | 96.57 | 46.2 | 54.03 | |
| TrustAlign-7BTraining Paradigm=RL based, Backbone=TrustAlign-7B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 0 | 34.02 | 24.32 | 0 | 86.76 | 49.8 | 55.54 | |
| TrustAlign-8BTraining Paradigm=RL based, Backbone=TrustAlign-8B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 0 | 30.57 | 22.64 | 88.73 | 88.73 | 49.6 | 55.69 |