Short-form generation on Short-form generation 1D-SameTask (OOD)
0.64PRRMSP
Evaluation results
| Method | Links | |
|---|---|---|
| MSPModel Backbone=Gemma-2-9B2026.04 | 0.64 | |
| MSPModel Backbone=Llama 3.1-8B2026.04 | 0.57 | |
| SATRMD-MSPModel Backbone=Gemma-2-9B2026.04 | 0.41 | |
| SAPLMA (middle)Model Backbone=Llama 3.1-8B2026.04 | 0.3 | |
| Uhead (1 layer)Model Backbone=Llama 3.1-8B2026.04 | 0.24 | |
| HUQ-SATRMDModel Backbone=Gemma-2-9B2026.04 | 0.23 | |
| SATRMDModel Backbone=Gemma-2-9B2026.04 | 0.21 | |
| Uhead (1 layer)Model Backbone=Gemma-2-9B2026.04 | 0.18 | |
| Uhead (2 layers)Model Backbone=Gemma-2-9B2026.04 | 0.18 | |
| Uhead (2 layers)Model Backbone=Llama 3.1-8B2026.04 | 0.15 | |
| SAPLMA (middle)Model Backbone=Gemma-2-9B2026.04 | 0.11 | |
| Lookback-lensModel Backbone=Gemma-2-9B2026.04 | 0.09 | |
| SATRMD-MSPModel Backbone=Llama 3.1-8B2026.04 | 0.05 | |
| SATMD-MSPModel Backbone=Llama 3.1-8B2026.04 | 0.04 | |
| Lookback-lensModel Backbone=Llama 3.1-8B2026.04 | -0.03 | |
| SAPLMA (top)Model Backbone=Gemma-2-9B2026.04 | -0.03 | |
| SATMD-MSPModel Backbone=Gemma-2-9B2026.04 | -0.07 | |
| HUQ-SATRMDModel Backbone=Llama 3.1-8B2026.04 | -0.08 | |
| SATRMDModel Backbone=Llama 3.1-8B2026.04 | -0.11 | |
| SAPLMA (top)Model Backbone=Llama 3.1-8B2026.04 | -0.12 | |
| HUQ-SATMDModel Backbone=Llama 3.1-8B2026.04 | -0.12 | |
| SATMDModel Backbone=Llama 3.1-8B2026.04 | -0.15 | |
| HUQ-SATMDModel Backbone=Gemma-2-9B2026.04 | -0.35 | |
| SATMDModel Backbone=Gemma-2-9B2026.04 | -0.39 |