Multiple-Answer Question Answering on QAMPARI
50.5PrecisionSPADER (QAMPARI-only)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SPADER (QAMPARI-only)Backbone=Qwen3-8B2026.05 | 50.5 | — | — | — | 33.6 | 37.8 | |
| SPADERBackbone=Qwen3-8B2026.05 | 47.4 | — | — | — | 31.2 | 34.3 | |
| SPADERBackbone=Llama3.1-8B2026.05 | 45.8 | — | — | — | 34 | 34.8 | |
| StepSearchBackbone=Qwen3-8B2026.05 | 42.9 | — | — | — | 30.4 | 31.7 | |
| GRPOBackbone=Qwen3-8B2026.05 | 42.3 | — | — | — | 18.2 | 22.6 | |
| SPADER (QAMPARI-only)Backbone=Llama3.1-8B2026.05 | 42 | — | — | — | 41.1 | 39.4 | |
| ReActBackbone=Qwen3-8B2026.05 | 41.9 | — | — | — | 17.3 | 21.8 | |
| StepSearchBackbone=Llama3.1-8B2026.05 | 41.8 | — | — | — | 25.8 | 30.4 | |
| R3-RAGBackbone=Qwen3-8B2026.05 | 41.7 | — | — | — | 25.4 | 27.6 | |
| PPOBackbone=Qwen3-8B2026.05 | 41 | — | — | — | 25.6 | 27.9 | |
| ReActBackbone=Llama3.1-8B2026.05 | 38.8 | — | — | — | 20.2 | 24.1 | |
| R3-RAGBackbone=Llama3.1-8B2026.05 | 38.6 | — | — | — | 29 | 30.3 | |
| PPOBackbone=Llama3.1-8B2026.05 | 35.3 | — | — | — | 25.3 | 26.8 | |
| RAGBackbone=Llama3.1-8B2026.05 | 33.6 | — | — | — | 11.8 | 15.7 | |
| GRPOBackbone=Llama3.1-8B2026.05 | 23.3 | — | — | — | 18 | 18.7 | |
| RAGBackbone=Qwen3-8B2026.05 | 18.1 | — | — | — | 11.9 | 12.8 | |
| Rewarding doubtMode=Fine-tuned2025.03 | — | 0.0816 | 0.6947 | 24.8 | — | — | |
| Sequence probabilityMode=Zero-shot2025.03 | — | 0.5324 | 0.5942 | 19.28 | — | — | |
| Trained probeMode=Supervised fine-tuning2025.03 | — | 0.1117 | 0.6481 | 22.33 | — | — | |
| VerbalizeMode=Zero-shot2025.03 | — | 0.5319 | 0.6047 | 25.5 | — | — |