Retrieval-Augmented Question Answering on NQ, TriviaQA, and WebQ (test)
62.2Answer Quality RecallERA
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ERABackbone=Qwen3-8B-Base2026.02 | 62.2 | 53.2 | 57.3 | 60.6 | 45.4 | 40.8 | 71.5 | 52 | 54.5 | |
| ERABackbone=Llama3.1-8B2026.02 | 53.7 | 68 | 60 | 50.4 | 36.7 | 75.9 | 50.6 | 60.7 | 60.4 | |
| LogprobBackbone=Llama3.1-8B, t=-22026.02 | 50.9 | 91 | 65.3 | 56.8 | 39.2 | 31.4 | 42.4 | 36.1 | 46.5 | |
| DTABackbone=Llama3.1-8B2026.02 | 49 | 54.4 | 51.6 | 51 | 32.2 | 59.6 | 48.1 | 53.3 | 52.4 | |
| ICLBackbone=Llama3.1-8B2026.02 | 47.4 | 33.6 | 39.4 | 54.8 | 32.6 | 1.4 | 80 | 2.7 | 5 | |
| DTABackbone=Qwen3-8B-Base2026.02 | 43.6 | 41 | 42.2 | 38 | 31.9 | 47.6 | 56.6 | 51.7 | 46.5 | |
| Self-ConsistencyBackbone=Llama3.1-8B2026.02 | 38.6 | 49.2 | 43.2 | 39.2 | 30 | 55.9 | 36.8 | 44.4 | 43.8 | |
| ptrueBackbone=Llama3.1-8B, t=0.52026.02 | 36.4 | 93.1 | 52.3 | 37.5 | 23.5 | 31.1 | 55.6 | 26.8 | 31.2 | |
| ptrueBackbone=Llama3.1-8B, t=0.72026.02 | 33.7 | 93.4 | 49.5 | 35.2 | 20.7 | 49.7 | 48.4 | 29.3 | 36.8 | |
| LogprobBackbone=Qwen3-8B-Base, t=-22026.02 | 30.8 | 93.3 | 46.3 | 30.3 | 22.1 | 5 | 56.4 | 9.1 | 15.3 | |
| LogprobBackbone=Llama3.1-8B, t=-12026.02 | 28.2 | 91.4 | 43.1 | 34.6 | 19.9 | 77.5 | 34 | 47.2 | 45.1 | |
| ptrueBackbone=Llama3.1-8B, t=0.92026.02 | 25.3 | 95.2 | 40 | 30 | 13 | 77.9 | 40.2 | 22.2 | 25.5 | |
| ptrueBackbone=Qwen3-8B-Base, t=0.52026.02 | 20.9 | 93.1 | 34.2 | 24.5 | 12.8 | 52.1 | 39.9 | 45.2 | 38.9 | |
| Self-ConsistencyBackbone=Qwen3-8B-Base2026.02 | 20.3 | 41.5 | 27.3 | 21.9 | 14.2 | 77.7 | 35 | 48.3 | 34.9 | |
| LogprobBackbone=Qwen3-8B-Base, t=-12026.02 | 19.7 | 93.3 | 32.5 | 18.7 | 14.4 | 68.9 | 36 | 47.3 | 38.5 | |
| ICLBackbone=Qwen3-8B-Base2026.02 | 17.8 | 13 | 15 | 18.4 | 14.2 | 4.8 | 40.4 | 8.5 | 10.9 | |
| ptrueBackbone=Qwen3-8B-Base, t=0.72026.02 | 13.3 | 97.3 | 23.4 | 16.1 | 3.6 | 88.3 | 35.3 | 50.4 | 32 | |
| ptrueBackbone=Qwen3-8B-Base, t=0.92026.02 | 1.4 | 100 | 2.9 | 1.4 | 0.1 | 99 | 29.4 | 45.3 | 5.4 |