Digit Span on Digit Span
1Human-Model SimilarityTASKPR
Evaluation Results
| Method | Links | |
|---|---|---|
| TASKPRModel=Claude Opus 4.62026.05 | 1 | |
| HUMPRModel=Llama 3 8B2026.05 | 1 | |
| COMPACTORModel=GPT-5.42026.05 | 0.95 | |
| MEMPRModel=Qwen3 8B NT2026.05 | 0.665 | |
| COMPACTORModel Backbone=Llama 3.3 70B2026.05 | 0.607 | |
| COMPACTORModel Backbone=Qwen3 8B T2026.05 | 0.607 | |
| COMPACTORModel Backbone=Qwen3 8B NT2026.05 | 0.586 | |
| COMPACTORModel Backbone=Qwen3 30B Think2026.05 | 0.551 | |
| COMPACTORModel Backbone=Qwen3 Next 80B2026.05 | 0.551 | |
| MEMPRModel Backbone=Qwen3 30B Think2026.05 | 0.55 | |
| MEMPRModel Backbone=Qwen3 8B T2026.05 | 0.387 | |
| COMPACTORModel Backbone=Llama 3 8B2026.05 | 0.353 | |
| MEMPRModel Backbone=Qwen3 8B NT2026.05 | 0.353 | |
| HumanModel=Human2026.05 | 0.344 | |
| TASKPRModel Backbone=Claude Opus 4.62026.05 | 0.324 | |
| TASKPRModel Backbone=Llama 3 8B2026.05 | 0.324 | |
| TASKPRModel Backbone=Llama 3.3 70B2026.05 | 0.324 | |
| TASKPRModel Backbone=GPT-5.42026.05 | 0.324 | |
| TASKPRModel Backbone=Qwen3 8B NT2026.05 | 0.324 | |
| TASKPRModel Backbone=Qwen3 8B T2026.05 | 0.324 | |
| TASKPRModel Backbone=Qwen3 30B Inst2026.05 | 0.324 | |
| TASKPRModel Backbone=Qwen3 30B Think2026.05 | 0.324 | |
| TASKPRModel Backbone=Qwen3 Next 80B2026.05 | 0.324 | |
| COMPACTORModel Backbone=GPT-5.42026.05 | 0.05 | |
| COMPACTORModel Backbone=Qwen3 30B Inst2026.05 | 0.05 | |
| MEMPRModel Backbone=Qwen3 Next 80B2026.05 | 0.03 | |
| HUMPRModel Backbone=Qwen3 30B Think2026.05 | 0.005 | |
| HUMPRModel Backbone=Claude Opus 4.62026.05 | 0 | |
| MEMPRModel Backbone=Claude Opus 4.62026.05 | 0 | |
| COMPACTORModel Backbone=Claude Opus 4.62026.05 | 0 | |
| HUMPRModel Backbone=Llama 3 8B2026.05 | 0 | |
| MEMPRModel Backbone=Llama 3 8B2026.05 | 0 | |
| HUMPRModel Backbone=Llama 3.3 70B2026.05 | 0 | |
| MEMPRModel Backbone=Llama 3.3 70B2026.05 | 0 | |
| HUMPRModel Backbone=GPT-5.42026.05 | 0 | |
| MEMPRModel Backbone=GPT-5.42026.05 | 0 | |
| HUMPRModel Backbone=Qwen3 8B NT2026.05 | 0 | |
| HUMPRModel Backbone=Qwen3 8B T2026.05 | 0 | |
| HUMPRModel Backbone=Qwen3 30B Inst2026.05 | 0 | |
| MEMPRModel Backbone=Qwen3 30B Inst2026.05 | 0 | |
| HUMPRModel Backbone=Qwen3 Next 80B2026.05 | 0 | |
| COMPACTORModel=Llama 3 8B2026.05 | 0 |