Question Answering on SciQ (Accuracy, Normalized Accuracy)
90.2AccuracyAIRAhybrid-C
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AIRAhybrid-CShot=0-shot, Architecture Variant=Stretched2026.05 | 90.2 | 85.4 | |
| AIRAhybrid-BShot=0-shot, Architecture Variant=Stacked2026.05 | 89.9 | 84.1 | |
| AIRAhybrid-DShot=0-shot, Architecture Variant=Stacked2026.05 | 89.9 | 85.8 | |
| AIRAhybrid-EShot=0-shot, Architecture Variant=Stretched2026.05 | 89.8 | 85.8 | |
| AIRAhybrid-AShot=0-shot, Architecture Variant=Stretched2026.05 | 89.7 | 85.2 | |
| Composer (2Mb-M-3A)Shot=0-shot2026.05 | 89.6 | 85.3 | |
| AIRAhybrid-BShot=0-shot, Architecture Variant=Stretched2026.05 | 89.5 | 85.7 | |
| AIRAhybrid-EShot=0-shot, Architecture Variant=Stacked2026.05 | 88.9 | 85.3 | |
| Mamba (Mb + M)Shot=0-shot2026.05 | 88.8 | 85.1 | |
| Nemotron-2 Approx.Shot=0-shot2026.05 | 88.7 | 84.3 | |
| AIRAhybrid-DShot=0-shot, Architecture Variant=Stretched2026.05 | 88.6 | 84.2 | |
| Nemotron-H Approx.Shot=0-shot2026.05 | 87.8 | 82.7 |