Question Answering on Natural Questions (PPL)
1.272PPLHourglass
Evaluation results
| Method | Links | |
|---|---|---|
| HourglassSize=1074M2026.02 | 1.272 | |
| ConventionalSize=1074M2026.02 | 1.311 | |
| Conventional (OLMo-2)Size=1074M2026.02 | 1.323 | |
| HourglassSize=906M2026.02 | 1.324 | |
| ConventionalSize=906M2026.02 | 1.325 | |
| HourglassSize=403M2026.02 | 1.386 | |
| ConventionalSize=403M2026.02 | 1.427 | |
| HourglassSize=113M2026.02 | 1.537 | |
| ConventionalSize=113M2026.02 | 1.604 |