Question Answering on XQuAD 1.0 (test)
79.6F1 ScoremLUKE-W
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| mLUKE-W#Params=561M, MRC Formulation=false2023.05 | 79.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mPMR#Params=550M, MRC Formulation=true2023.05 | 79.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 64.4 | |
| VECO#Params=550M, MRC Formulation=false2023.05 | 77.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 61.8 | |
| KMLM#Params=550M, MRC Formulation=false2023.05 | 77.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 61.7 | |
| XLM-R#Params=550M, MRC Formulation=true2023.05 | 77.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 61.3 | |
| XLM-R#Params=550M, MRC Formulation=false2023.05 | 76.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 60.8 | |
| mPMR_base#Params=270M, MRC Formulation=true2023.05 | 74 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 59.5 | |
| Wiki-CL#Params=550M, MRC Formulation=false2023.05 | 72.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 56.9 | |
| XLM-R_base#Params=270M, MRC Formulation=true2023.05 | 70.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 56.9 | |
| mT5#Params=580M, MRC Formulation=false2023.05 | 67 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 49 | |
| Llama baseBackbone=Llama, Method=base2026.01 | — | 23.2 | 38.54 | 25.18 | 23.47 | 34.23 | 29.47 | 30.01 | 27.32 | 19.23 | 23.74 | 34.91 | 12.84 | 26.85 | 7.43 | — | |
| Llama CLASBackbone=Llama, Method=CLAS2026.01 | — | 1.01 | 6.25 | 0.23 | 0 | 1.84 | -0.1 | -0.03 | -0.06 | -0.46 | 1.14 | 1.67 | -0.25 | 0.94 | 8.83 | — | |
| Llama Int μBackbone=Llama, Method=Int μ2026.01 | — | -1.95 | -5.46 | 1.5 | 0 | 1.08 | -0.09 | 6.97 | -0.08 | -3.4 | -2.11 | 5.08 | 0.05 | 0.13 | 8.75 | — | |
| Qwen baseBackbone=Qwen, Method=base2026.01 | — | 28.73 | 34.77 | 37.06 | 48.88 | 38.53 | 33.12 | 34.48 | 27.58 | 28.51 | 29.19 | 31.67 | 20.24 | 32.73 | 5.16 | — | |
| Qwen CLASBackbone=Qwen, Method=CLAS2026.01 | — | 5.06 | -0.96 | -1.51 | 0 | 2.63 | -5.28 | 0.48 | 8.94 | 0.98 | -2.31 | 1.36 | 3.76 | 1.1 | 4.95 | — | |
| Qwen Int μBackbone=Qwen, Method=Int μ2026.01 | — | 0.26 | -2.76 | -0.06 | 0.11 | -2.53 | -2.12 | -0.48 | -0.58 | 7.48 | -2.19 | -5.67 | -1.24 | -0.82 | 5.45 | — |