Commonsense Reasoning on OpenBookQA (normalized accuracy)
51.4Normalized Accuracy (OpenBookQA)Mistral-Large-3 675B-Base-2512
Evaluation results
| Method | Links | |
|---|---|---|
| Mistral-Large-3 675B-Base-2512shots=0-shot2026.06 | 51.4 | |
| Kimi-K2 Baseshots=0-shot2026.06 | 50.8 | |
| N-3-Super 120B-A12B-BaseShots=02026.04 | 50.2 | |
| GLM-4.5 Baseshots=0-shot2026.06 | 49.6 | |
| GLM-4.5 Air-BaseShots=02026.04 | 48.6 | |
| Nemotron-3-Ultra 550B-A55B-Baseshots=0-shot2026.06 | 48.6 | |
| DeepSeek-V3.2 Exp-Baseshots=0-shot2026.06 | 48.2 | |
| Ling-flash base-2.0Shots=02026.04 | 47 | |
| SwiAttnAttention Mechanism=Switch Attention2026.03 | 36.8 | |
| StaticHybridAttention Mechanism=Static hybrid of full attention and SWA, Ratio (Full:SWA)=1:32026.03 | 36.7 | |
| SWA-ZSAttention Mechanism=Sliding Window Attention, Training/Evaluation Protocol=Zero-shot2026.03 | 35.4 | |
| FullAttnAttention Mechanism=Full attention2026.03 | 35.4 | |
| SWA-CPTAttention Mechanism=Sliding Window Attention, Training/Evaluation Protocol=Continual pretraining2026.03 | 35.2 |