Reading Comprehension on Quoref
69.42F1 ScoreOLTQA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OLTQAConfiguration=Full2023.05 | 69.42 | — | |
| OLTQAKnowledge Distillation=Back KD2023.05 | 69.18 | — | |
| OLTQAKnowledge Distillation=Static MKD2023.05 | 69.12 | — | |
| OLTQAKnowledge Distillation=without MKD2023.05 | 69 | — | |
| OLTQARetriever=EPR2023.05 | 68.73 | — | |
| OLTQARetriever=BM252023.05 | 68.09 | — | |
| OLTQAAblation=without Pm2023.05 | 67.2 | — | |
| OLTQARe-ranker=excluded2023.05 | 65.38 | — | |
| Hyperformer++2023.05 | 60.8 | — | |
| Muppet2023.05 | 57.66 | — | |
| OLTQAAblation=without Pk2023.05 | 56.32 | — | |
| UnifiedQA2023.05 | 56.28 | — | |
| EPR2023.05 | 48.54 | — | |
| B-2Backbone=LLaMA-3.1-8B, Inference Setting=Zero-shot2026.04 | 46.2 | 36.7 | |
| SAVERBackbone=LLaMA-3.1-8B, Inference Setting=Zero-shot2026.04 | 45.7 | 37.2 | |
| MADBackbone=LLaMA-3.1-8B, Inference Setting=Zero-shot2026.04 | 45.2 | 36.3 | |
| MADBackbone=Qwen-2.5-7B, Inference Setting=Zero-shot2026.04 | 44.8 | 35.3 | |
| SAVERBackbone=Qwen-2.5-7B, Inference Setting=Zero-shot2026.04 | 44.1 | 35.6 | |
| Self-RefineBackbone=LLaMA-3.1-8B, Inference Setting=Zero-shot2026.04 | 43.6 | 34.1 | |
| B-2Backbone=Qwen-2.5-7B, Inference Setting=Zero-shot2026.04 | 43.3 | 35.1 | |
| Self-RefineBackbone=Qwen-2.5-7B, Inference Setting=Zero-shot2026.04 | 42.9 | 33.8 | |
| CoTBackbone=Qwen-2.5-7B, Inference Setting=Zero-shot2026.04 | 42.5 | 32.8 | |
| CoTBackbone=LLaMA-3.1-8B, Inference Setting=Zero-shot2026.04 | 42.3 | 33.2 | |
| SAVERBackbone=LLaMA-3.2-3B, Inference Setting=Zero-shot2026.04 | 41.9 | 33.2 | |
| MADBackbone=LLaMA-3.2-3B, Inference Setting=Zero-shot2026.04 | 41.2 | 33.5 | |
| B-2Backbone=LLaMA-3.2-3B, Inference Setting=Zero-shot2026.04 | 40.7 | 32.1 | |
| Self-RefineBackbone=LLaMA-3.2-3B, Inference Setting=Zero-shot2026.04 | 39.4 | 29.9 | |
| CoTBackbone=LLaMA-3.2-3B, Inference Setting=Zero-shot2026.04 | 37.8 | 29.2 | |
| Vanilla LMBackbone=LLaMA-3.1-8B, Inference Setting=Zero-shot2026.04 | 37.7 | 29.4 | |
| Vanilla LMBackbone=Qwen-2.5-7B, Inference Setting=Zero-shot2026.04 | 36.5 | 28.4 | |
| ProQA2023.05 | 35.75 | — | |
| Vanilla LMBackbone=LLaMA-3.2-3B, Inference Setting=Zero-shot2026.04 | 34.6 | 24.4 |