Long-context language modeling on ZeroSCROLLS (test)
35.8GovReport ScoreGDWM
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GDWMModel=QWEN3-4B, Steps=322026.01 | 35.8 | 9.2 | 36.5 | 39.8 | 87.5 | 32.8 | 40.3 | |
| GDWMModel=QWEN3-4B, Steps=162026.01 | 35 | 8.8 | 35.8 | 38.6 | 86.8 | 32 | 39.5 | |
| GDWMModel=QWEN3-4B, Steps=82026.01 | 34.2 | 8.4 | 35.2 | 37.5 | 82.2 | 31.2 | 38.1 | |
| qTTTModel=QWEN3-4B, Steps=322026.01 | 33.2 | 8.9 | 33.8 | 38.4 | 87.2 | 30.8 | 38.7 | |
| qTTTModel=QWEN3-4B, Steps=162026.01 | 31.9 | 8.6 | 32.1 | 35.3 | 84.8 | 27.5 | 36.7 | |
| GDWMModel=QWEN3-8B, Steps=322026.01 | 31.5 | 9 | 28.8 | 44.2 | 94.8 | 51 | 43.2 | |
| GDWMModel=QWEN3-8B, Steps=162026.01 | 30.8 | 8.6 | 28.1 | 43.1 | 94.3 | 50.2 | 42.5 | |
| GDWMModel=QWEN3-8B, Steps=82026.01 | 30.2 | 8.2 | 27.5 | 42 | 93.8 | 49.5 | 41.9 | |
| GDWMModel=QWEN3-1.7B, Steps=322026.01 | 29.8 | 10.2 | 35.5 | 13 | 77.5 | 28.8 | 32.5 | |
| qTTTModel=QWEN3-8B, Steps=322026.01 | 29.8 | 9 | 27 | 42.4 | 94.9 | 49.6 | 42.2 | |
| qTTTModel=QWEN3-4B, Steps=82026.01 | 29.2 | 8.3 | 29.5 | 32.3 | 81.3 | 23.7 | 34 | |
| GDWMModel=QWEN3-1.7B, Steps=162026.01 | 29.1 | 9.9 | 34.6 | 12.6 | 77 | 27.9 | 31.8 | |
| GDWMModel=QWEN3-1.7B, Steps=82026.01 | 28.5 | 9.6 | 33.8 | 12.2 | 76.5 | 27 | 31.3 | |
| qTTTModel=QWEN3-8B, Steps=162026.01 | 27.9 | 8.7 | 24.5 | 40.6 | 93.1 | 46.2 | 40.2 | |
| qTTTModel=QWEN3-1.7B, Steps=322026.01 | 26.8 | 9.7 | 31.5 | 12.4 | 76.5 | 26.4 | 30.6 | |
| qTTTModel=QWEN3-8B, Steps=82026.01 | 25.3 | 8.5 | 22.8 | 38.5 | 91.2 | 42 | 38.1 | |
| qTTTModel=QWEN3-1.7B, Steps=162026.01 | 25.1 | 9.3 | 29.5 | 11.2 | 74.1 | 23.1 | 28.7 | |
| In-contextModel=QWEN3-4B, Steps=N/A2026.01 | 24.8 | 11.2 | 23.5 | 10.9 | 40.8 | 17 | 21.4 | |
| qTTTModel=QWEN3-1.7B, Steps=82026.01 | 23.9 | 8.3 | 27.3 | 9.4 | 72.1 | 19.9 | 26.8 | |
| In-contextModel=QWEN3-8B, Steps=N/A2026.01 | 22.5 | 8.8 | 20.1 | 35.4 | 90.5 | 22.9 | 33.4 | |
| In-contextModel=QWEN3-1.7B, Steps=N/A2026.01 | 22.2 | 6.4 | 25.8 | 14.9 | 48.1 | 11.8 | 21.5 | |
| ThinkingModel=QWEN3-1.7B, Steps=N/A2026.01 | 21.5 | 7.6 | 22.3 | 9.2 | 61.8 | 22.4 | 24.1 | |
| ThinkingModel=QWEN3-4B, Steps=N/A2026.01 | 20.8 | 7.5 | 25.2 | 29.8 | 76.1 | 8.3 | 28 | |
| ThinkingModel=QWEN3-8B, Steps=N/A2026.01 | 18.2 | 9.8 | 21.5 | 19.6 | 71.8 | 43.8 | 30.8 |