Reading Comprehension on BoolQ and DROP
81.1BoolQ AccuracyOLMoE-1B-7B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OLMoE-1B-7BNact/Ntotal=1.3B/6.9B, Model Type=Instruct-tuned2026.05 | 81.1 | 29.3 | |
| MobileMoE-LNact/Ntotal=922M/5.3B, Model Type=Instruct-tuned2026.05 | 81.1 | 50.1 | |
| MobileMoE-MNact/Ntotal=528M/2.8B, Model Type=Instruct-tuned2026.05 | 76.7 | 46.6 | |
| Gemma 3 1BNact/Ntotal=1.0B, Model Type=Instruct-tuned2026.05 | 75.8 | 22 | |
| MobileMoE-SNact/Ntotal=272M/1.3B, Model Type=Instruct-tuned2026.05 | 72.3 | 32.3 | |
| Qwen3.5 2BNact/Ntotal=1.9B, Model Type=Instruct-tuned2026.05 | 71.7 | 44.7 | |
| Llama 3.2 1BNact/Ntotal=1.2B, Model Type=Instruct-tuned2026.05 | 71.6 | 21.5 | |
| OLMo 2 1BNact/Ntotal=1.5B, Model Type=Instruct-tuned2026.05 | 71 | 31.2 | |
| MobileLLM-ProNact/Ntotal=1.1B, Model Type=Instruct-tuned2026.05 | 68.8 | 25.5 | |
| SmolLM2 1.7BNact/Ntotal=1.7B, Model Type=Instruct-tuned2026.05 | 68.5 | 24.8 | |
| Qwen3.5 0.8BNact/Ntotal=749M, Model Type=Instruct-tuned2026.05 | 62.5 | 33.3 | |
| Gemma 3 270MNact/Ntotal=270M, Model Type=Instruct-tuned2026.05 | 56.1 | 11 | |
| SmolLM2 360MNact/Ntotal=362M, Model Type=Instruct-tuned2026.05 | 42.5 | 15.2 |