Reasoning on MMLU Redux
67.3AccuracyQwen3-1.7B-ALLMEM
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen3-1.7B-ALLMEMModel Size=1.7B, Architecture=ALLMEM, Sliding window length=2048, Attention sinks=128, Few-shot setting=0-shot, Max tokens=327682026.02 | 67.3 | |
| Qwen3-1.7BModel Size=1.7B, Architecture=Standard, Few-shot setting=0-shot, Max tokens=327682026.02 | 66.5 | |
| Qwen3-0.6B-ALLMEMModel Size=0.6B, Architecture=ALLMEM, Sliding window length=2048, Attention sinks=128, Few-shot setting=0-shot, Max tokens=327682026.02 | 47.05 | |
| Qwen3-0.6BModel Size=0.6B, Architecture=Standard, Few-shot setting=0-shot, Max tokens=327682026.02 | 44.9 |