Question Answering on PopQA (EM, F1, LLM)
46.1EMIGPO (gold-sup.)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| IGPO (gold-sup.)Backbone=Qwen3-8B2026.05 | 46.1 | 49.8 | — | |
| IGPO (gold-sup.)Backbone=Qwen3-4B2026.05 | 44.9 | 48.6 | — | |
| SIOPBackbone=Qwen3-8B2026.05 | 44.5 | 47.9 | — | |
| GRPO (gold-sup.)Backbone=Qwen3-4B2026.05 | 43.6 | 47.8 | — | |
| GRPO (gold-sup.)Backbone=Qwen3-8B2026.05 | 41.8 | 47.4 | — | |
| TTRLBackbone=Qwen3-4B2026.05 | 41.6 | 45.8 | — | |
| SIOPBackbone=Qwen3-4B2026.05 | 41.6 | 46.5 | — | |
| TTRLBackbone=Qwen3-8B2026.05 | 41.6 | 47.6 | — | |
| EMPOBackbone=Qwen3-8B2026.05 | 41.5 | 45 | — | |
| EMPOBackbone=Qwen3-4B2026.05 | 41.2 | 46 | — | |
| LLM (with RAG*)Backbone=Mistral-7B-Instruct, Top-k=5, Comp. Rate=-2026.01 | 34.2 | 40.17 | 41.05 | |
| SeleComBackbone=Mistral-7B-Instruct, Top-k=5, Comp. Rate=82x2026.01 | 33.32 | 40.51 | 39.56 | |
| LLM (with RAG*)Backbone=Qwen2.5-7B-Instruct, Top-k=5, Comp. Rate=-2026.01 | 32.28 | 38.52 | 37.92 | |
| SeleComBackbone=Qwen2.5-7B-Instruct, Top-k=5, Comp. Rate=82x2026.01 | 32.23 | 38.71 | 39.02 | |
| SeleComBackbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=82x2026.01 | 31.42 | 37.61 | 37.65 | |
| SeleComBackbone=Qwen2.5-7B-Instruct, Top-k=1, Comp. Rate=82x2026.01 | 31.2 | 37.71 | 37.04 | |
| LLM (with RAG*)Backbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=-2026.01 | 30.01 | 36.27 | 35.83 | |
| LLM (with RAG*)Backbone=Qwen2.5-7B-Instruct, Top-k=1, Comp. Rate=-2026.01 | 29.33 | 35.77 | 35.13 | |
| ICAEBackbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=4x2026.01 | 26.84 | 33.8 | 36.71 | |
| LLM (with RAG)Backbone=Qwen2.5-7B-Instruct, Top-k=1, Comp. Rate=-2026.01 | 25.22 | 33.19 | 38.09 | |
| COCOMBackbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=128x2026.01 | 21.72 | 26.21 | 25.32 | |
| LLM (with RAG)Backbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=-2026.01 | 20.91 | 29.26 | 38.9 | |
| LLMLingua-2Backbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=4x2026.01 | 14.02 | 21.29 | 29.68 | |
| LLM (w/o RAG)Backbone=Qwen2.5-7B-Instruct, Top-k=1, Comp. Rate=-2026.01 | 10.57 | 15.84 | 18.16 | |
| xRAGBackbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=164x2026.01 | 10.06 | 18.3 | 36.66 | |
| LLM (w/o RAG)Backbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=-2026.01 | 9.4 | 16.06 | 22.58 | |
| PISCOBackbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=64x2026.01 | 0.44 | 13.28 | 38.38 |