Early-stopping for mathematical reasoning on 5K corpus 1.0 (test)
67.4Savings RatioTTT QK (dh=128)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| TTT QK (dh=128)Label mode=Supervised labels, delta=0.2, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 67.4 | 20.4 | |
| TTT no-QKLabel mode=Supervised labels, delta=0.2, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 67.3 | 19.2 | |
| TTT no-QKLabel mode=Consistent labels (no ground truth), delta=0.2, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 64.4 | 19.3 | |
| TTT QK (dh=128)Label mode=Consistent labels (no ground truth), delta=0.2, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 62.9 | 18.7 | |
| Static ProbeLabel mode=Supervised labels, delta=0.2, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 62.5 | 20.8 | |
| TTT no-QKLabel mode=Supervised labels, delta=0.15, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 57.5 | 15.2 | |
| Static ProbeLabel mode=Consistent labels (no ground truth), delta=0.2, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 57.3 | 19.7 | |
| TTT QK (dh=128)Label mode=Supervised labels, delta=0.15, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 56 | 15 | |
| TTT no-QKLabel mode=Consistent labels (no ground truth), delta=0.15, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 52.9 | 14.1 | |
| TTT QK (dh=128)Label mode=Consistent labels (no ground truth), delta=0.15, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 52.4 | 15 | |
| Static ProbeLabel mode=Supervised labels, delta=0.15, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 51.2 | 15.9 | |
| Static ProbeLabel mode=Consistent labels (no ground truth), delta=0.15, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 48.3 | 15.6 | |
| TTT no-QKLabel mode=Supervised labels, delta=0.1, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 47.5 | 11 | |
| TTT QK (dh=128)Label mode=Supervised labels, delta=0.1, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 41.4 | 10.3 | |
| TTT no-QKLabel mode=Consistent labels (no ground truth), delta=0.1, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 40.7 | 9.6 | |
| TTT QK (dh=128)Label mode=Consistent labels (no ground truth), delta=0.1, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 39.7 | 11.3 | |
| Static ProbeLabel mode=Supervised labels, delta=0.1, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 38 | 10.5 | |
| Static ProbeLabel mode=Consistent labels (no ground truth), delta=0.1, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 34.5 | 9.8 | |
| TTT no-QKLabel mode=Supervised labels, delta=0.05, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 28.2 | 5.3 | |
| TTT QK (dh=128)Label mode=Supervised labels, delta=0.05, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 23.3 | 4.6 | |
| TTT QK (dh=128)Label mode=Consistent labels (no ground truth), delta=0.05, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 23.2 | 6.4 | |
| Static ProbeLabel mode=Supervised labels, delta=0.05, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 22 | 5.5 | |
| TTT no-QKLabel mode=Consistent labels (no ground truth), delta=0.05, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 22 | 4.5 | |
| Static ProbeLabel mode=Consistent labels (no ground truth), delta=0.05, Backbone=Qwen2.5-32B, epsilon=0.052026.04 | 16.6 | 4.9 |