Short-form Question Answering on SimpleQA
7.9ECEDINCO
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| DINCOModel=Gemini-2.5-Flash, Logit access=Black-box2025.09 | 7.9 | 19.9 | 69.7 | |
| DINCOModel=Gemini-2.5-Flash, Logit access=White-box2025.09 | 8.8 | 17.4 | 76.2 | |
| DINCOModel=GPT-4.1, Logit access=White-box2025.09 | 8.9 | 18.3 | 78.6 | |
| MSPModel=Gemini-2.5-Flash, Logit access=White-box2025.09 | 9.8 | 17.7 | 77.3 | |
| NVCModel=Gemini-2.5-Flash, Logit access=White-box2025.09 | 10.5 | 19.9 | 66.2 | |
| DINCOModel=GPT-4.1, Logit access=Black-box2025.09 | 16.1 | 25.1 | 60.5 | |
| NVCModel=GPT-4.1, Logit access=White-box2025.09 | 16.4 | 22.2 | 72.9 | |
| SCModel=Gemini-2.5-Flash, Logit access=White-box2025.09 | 17 | 21.2 | 74.8 | |
| SC-VCModel=Gemini-2.5-Flash, Logit access=White-box2025.09 | 18.6 | 22.1 | 75.5 | |
| NVCModel=Gemini-2.5-Flash, Logit access=Black-box2025.09 | 20.8 | 26.2 | 59.5 | |
| NVCModel=GPT-4.1, Logit access=Black-box2025.09 | 21.3 | 27 | 60.7 | |
| SCModel=GPT-4.1, Logit access=White-box2025.09 | 22 | 25.2 | 75 | |
| SC-VCModel=GPT-4.1, Logit access=White-box2025.09 | 22.3 | 25.2 | 76.1 | |
| MSPModel=GPT-4.1, Logit access=White-box2025.09 | 26.3 | 25.5 | 80 | |
| K-VCModel=GPT-4.1, Logit access=White-box2025.09 | 33.8 | 33.7 | 63.2 | |
| VCModel=Gemini-2.5-Flash, Logit access=White-box2025.09 | 40.9 | 39.3 | 61.7 | |
| K-VCModel=Gemini-2.5-Flash, Logit access=White-box2025.09 | 53.5 | 51.1 | 56.6 | |
| VCModel=GPT-4.1, Logit access=White-box2025.09 | 54.7 | 54.9 | 64.4 |