Long-context Reasoning and Retrieval on RULER-PT (aggregate) (1K-4K Context Sweep)
97.7RULER-PT (Aggregate) Score @ 1024 ContextQwen3-1.7B
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Qwen3-1.7Bparameters=1.7B2026.03 | 97.7 | 93 | 96.1 | |
| Qwen3-4Bparameters=4B2026.03 | 96.6 | 98.4 | 97.9 | |
| Qwen3-0.6Bparameters=0.6B2026.03 | 92.4 | 86.1 | 88.5 | |
| Tucano2-qwen-3.7B-Thinkparameters=3.7B, mode=Think, base_model=Qwen2026.03 | 81.7 | 76.5 | 70.7 | |
| Tucano2-qwen-3.7B-Instructparameters=3.7B, mode=Instruct, base_model=Qwen2026.03 | 79.5 | 71 | 68.6 | |
| Tucano2-qwen-0.5B-Instructparameters=0.5B, mode=Instruct, base_model=Qwen2026.03 | 60.7 | 48.1 | — | |
| Tucano2-qwen-1.5B-Thinkparameters=1.5B, mode=Think, base_model=Qwen2026.03 | 60.3 | 50.1 | 44 | |
| Tucano2-qwen-1.5B-Instructparameters=1.5B, mode=Instruct, base_model=Qwen2026.03 | 57.3 | 49.4 | 42.1 | |
| Tucano2-qwen-0.5B-Thinkparameters=0.5B, mode=Think, base_model=Qwen2026.03 | 47.1 | 36.4 | 29.3 |