Question Answering on QASPER 1200:251 (test)
28.92Answerable EMQwen3-4B
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Qwen3-4BTraining Paradigm=SFT based, Backbone=Qwen3-4B, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 28.92 | 69.53 | 62.25 | 67.33 | 67.33 | 63.13 | 64.79 | |
| Llama3-8B-InstructTraining Paradigm=SFT based, Backbone=Llama3-8B-Instruct, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 27.92 | 70.21 | 62.75 | 62.95 | 62.95 | 62.78 | 62.85 | |
| GRACE-Qwen3-4BTraining Paradigm=Our methods, Backbone=Qwen3-4B, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 27.08 | 73.09 | 74.25 | 74.9 | 74.9 | 74.36 | 74.58 | |
| GRACE-Llama3.1-8BTraining Paradigm=Our methods, Backbone=Llama3.1-8B, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 27.08 | 72.02 | 70.08 | 74.5 | 74.5 | 70.85 | 72.29 | |
| ARENA-8BTraining Paradigm=RL based, Backbone=ARENA-8B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 23.58 | 68.66 | 69.17 | 0 | 0.4 | 57.27 | 34.79 | |
| ARENA on Qwen3-4BTraining Paradigm=RL based, Method Strategy=ARENA, Backbone=Qwen3-4B, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 23.25 | 68.08 | 73.5 | 19.92 | 20.72 | 64.37 | 47.11 | |
| SimpleDeepSearcher-7BTraining Paradigm=Agentic Models, Backbone=SimpleDeepSearcher-7B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 22.92 | 67.13 | 67.17 | 16.73 | 17.93 | 58.65 | 42.55 | |
| ARENA-7BTraining Paradigm=RL based, Backbone=ARENA-7B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 22.5 | 69.73 | 69.67 | 4.78 | 7.97 | 58.99 | 38.82 | |
| Search-R1-7BTraining Paradigm=Agentic Models, Backbone=Search-R1-7B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 18.5 | 64.76 | 61 | 30.68 | 33.07 | 56.17 | 47.03 | |
| R1-Searcher-7BTraining Paradigm=Agentic Models, Backbone=R1-Searcher-7B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 15.42 | 59.8 | 48.83 | 17.13 | 19.92 | 43.83 | 34.38 | |
| SuRe on DeepSeek-chatTraining Paradigm=Prompt based, Method Strategy=SuRe, Backbone=DeepSeek-chat, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 13.08 | 58.68 | 57.92 | 0 | 7.17 | 49.14 | 32.54 | |
| SuRe on Qwen3-4BTraining Paradigm=Prompt based, Method Strategy=SuRe, Backbone=Qwen3-4B, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 5.92 | 52.43 | 46.17 | 0 | 0.4 | 38.25 | 23.28 | |
| SuRe on Llama3-8B-InstructTraining Paradigm=Prompt based, Method Strategy=SuRe, Backbone=Llama3-8B-Instruct, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 5.25 | 55.01 | 54.83 | 0 | 1.59 | 45.62 | 28.21 | |
| Llama3-8B-InstructTraining Paradigm=Prompt based, Backbone=Llama3-8B-Instruct, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 1.33 | 54.6 | 70.58 | 12.75 | 30.68 | 63.68 | 50.63 | |
| TrustAlign-8BTraining Paradigm=RL based, Backbone=TrustAlign-8B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 0.08 | 34.72 | 9.67 | 97.61 | 97.61 | 24.88 | 53.64 | |
| Qwen3-4BTraining Paradigm=Prompt based, Backbone=Qwen3-4B, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 0 | 50.23 | 78.83 | 5.58 | 42.23 | 72.5 | 60.53 | |
| DeepSeek-chatTraining Paradigm=Prompt based, Backbone=DeepSeek-chat, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 0 | 51.88 | 69.33 | 50.6 | 72.11 | 69.81 | 69.57 | |
| TrustAlign-3BTraining Paradigm=RL based, Backbone=TrustAlign-3B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 0 | 41.2 | 41.17 | 78.09 | 80.88 | 48.04 | 61.03 | |
| TrustAlign on Qwen3-4BTraining Paradigm=RL based, Method Strategy=TrustAlign, Backbone=Qwen3-4B, Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 0 | 44.5 | 36.83 | 0 | 60.96 | 41.01 | 48.9 | |
| TrustAlign-7BTraining Paradigm=RL based, Backbone=TrustAlign-7B, Checkpoint Status=released (†), Retriever Config=k=3, Embedding Model=Qwen3-Embedding-0.6B2026.01 | 0 | 43.63 | 33.67 | 0 | 88.45 | 43.14 | 38.41 |