Reasoning step reduction on In-Distribution 5K corpus (test)
47.5Savings RateTTT no-QK
Evaluation results
| Method | Links | ||
|---|---|---|---|
| TTT no-QKBackbone=Qwen2.5-32B, Supervision mode=supervised, delta=0.12026.04 | 47.5 | 11 | |
| TTT no-QKBackbone=Llama-3.3-70B, Supervision mode=supervised, delta=0.12026.04 | 42.4 | 9 | |
| TTT QKBackbone=Qwen2.5-32B, Supervision mode=supervised, delta=0.1, dh=1282026.04 | 41.4 | 10.3 | |
| TTT no-QKBackbone=QwQ-32B, Supervision mode=supervised, delta=0.12026.04 | 39.4 | 8.1 | |
| Static ProbeBackbone=Qwen2.5-32B, Supervision mode=supervised, delta=0.12026.04 | 38 | 10.5 | |
| TTT QKBackbone=Llama-3.3-70B, Supervision mode=supervised, delta=0.1, dh=1282026.04 | 37.8 | 8.1 | |
| TTT QKBackbone=QwQ-32B, Supervision mode=supervised, delta=0.1, dh=1282026.04 | 37.6 | 7.6 | |
| Static ProbeBackbone=Llama-3.3-70B, Supervision mode=supervised, delta=0.12026.04 | 35.4 | 10.4 | |
| Static ProbeBackbone=QwQ-32B, Supervision mode=supervised, delta=0.12026.04 | 29.5 | 9.4 |