Question Answering on HotpotQA (Adaptive Metrics Ac/Y)
84.5Accuracy (alpha=0.10)J-Q8B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| J-Q8BModel=Qwen3-8B2026.05 | 84.5 | 70.9 | 69.1 | 40 | 89.6 | 1.3 | |
| J-Q8BModel=Gemma-4-E4B2026.05 | 83.9 | 72.3 | 71.3 | 52.7 | 87.4 | 4.6 | |
| J-G26BModel=Gemma-4-E4B2026.05 | 83.8 | 71.9 | 71.2 | 53.5 | 85.1 | 20.6 | |
| R-L8BModel=Gemma-4-E4B2026.05 | 83.4 | 72.8 | 71.9 | 56.2 | 86.2 | 17.9 | |
| J-Q30BModel=Gemma-4-E4B2026.05 | 83.4 | 72.7 | 71.8 | 56.3 | 83.7 | 19.1 | |
| PplModel=Gemma-4-E4B2026.05 | 83.4 | 71.7 | 69.3 | 55.1 | 85.4 | 13.6 | |
| R-Q8BModel=Gemma-4-E4B2026.05 | 83.1 | 72.4 | 71.4 | 53.7 | 82 | 13.3 | |
| Ppl-StdModel=Gemma-4-E4B2026.05 | 83.1 | 71.8 | 68.8 | 54.7 | 80.7 | 14.9 | |
| MVModel=Gemma-4-E4B2026.05 | 82.8 | 72.4 | — | 33.3 | — | 0 | |
| Ppl-StdModel=Qwen3-8B2026.05 | 81.9 | 69.5 | 66.6 | 53.6 | 83.1 | 14.2 | |
| J-Q30BModel=Qwen3-8B2026.05 | 81.8 | 71 | 68.5 | 51.1 | 84.6 | 3 | |
| MVModel=Qwen3-8B2026.05 | 81.6 | 70.8 | — | 36.7 | — | 0 | |
| R-L8BModel=Qwen3-8B2026.05 | 81.6 | 70.4 | 69 | 53.3 | 83.2 | 13.1 | |
| PplModel=Qwen3-8B2026.05 | 81.6 | 69.6 | 66.9 | 53.6 | 82.9 | 10.5 | |
| R-Q8BModel=Qwen3-8B2026.05 | 81.4 | 70.5 | 68.8 | 54.7 | 77.8 | 11.2 | |
| J-G26BModel=Qwen3-8B2026.05 | 81.1 | 70.8 | 67.8 | 53.6 | 82.4 | 8.5 | |
| OracleModel=Gemma-4-E4B2026.05 | 79.1 | — | 79.1 | 100 | 79.1 | 100 | |
| OracleModel=Qwen3-8B2026.05 | 77.8 | — | 77.8 | 100 | 77.8 | 100 | |
| OracleModel=Qwen3-1.7B2026.05 | 73.9 | — | 73.9 | 100 | 73.9 | 100 | |
| GreedyModel=Gemma-4-E4B2026.05 | 70 | — | — | 100 | 70 | 100 | |
| MVModel=Qwen3-1.7B2026.05 | 68.6 | 51.9 | — | 24.4 | 72.2 | 12.7 | |
| J-Q8BModel=Qwen3-1.7B2026.05 | 68.6 | 52.2 | 45.2 | 26.6 | 71.2 | 13.3 | |
| J-Q30BModel=Qwen3-1.7B2026.05 | 68.3 | 51.1 | 45.5 | 27.2 | 70.9 | 14.2 | |
| PplModel=Qwen3-1.7B2026.05 | 68.1 | 48.5 | 40.8 | 30.2 | 72.8 | 15.5 | |
| GreedyModel=Qwen3-8B2026.05 | 67.6 | — | — | 100 | 67.6 | 100 | |
| Ppl-StdModel=Qwen3-1.7B2026.05 | 67.5 | 47.1 | 38.8 | 29.7 | 73.1 | 15 | |
| J-G26BModel=Qwen3-1.7B2026.05 | 67.4 | 50.5 | 39.8 | 27.8 | 70.1 | 15.1 | |
| R-L8BModel=Qwen3-1.7B2026.05 | 66.8 | 48.9 | 42.4 | 27 | 68.2 | 14.7 | |
| R-Q8BModel=Qwen3-1.7B2026.05 | 64.5 | 47.5 | 40.2 | 26.4 | 68 | 14.2 | |
| GreedyModel=Qwen3-1.7B2026.05 | 42.6 | — | — | 100 | 42.6 | 100 |