Multistep Soft Reasoning on MUSR (Accuracy %)
50.77Accuracy (Multi-choice)TMAP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| TMAPBackbone=Mistral-v0.2-7B2026.05 | 50.77 | — | |
| Data-WhispererBackbone=Mistral-v0.2-7B2026.05 | 47.89 | — | |
| BaseBackbone=Mistral-v0.2-7B2026.05 | 47.49 | — | |
| AlpagasusBackbone=Mistral-v0.2-7B2026.05 | 46.43 | — | |
| Full (100%)Backbone=Mistral-v0.2-7B2026.05 | 46.3 | — | |
| RandomBackbone=Mistral-v0.2-7B2026.05 | 46.16 | — | |
| Full (100%)Backbone=Qwen2.5-7B2026.05 | 46.03 | — | |
| LongestBackbone=Mistral-v0.2-7B2026.05 | 45.9 | — | |
| TMAPBackbone=Qwen2.5-7B2026.05 | 45.9 | — | |
| DEITABackbone=Mistral-v0.2-7B2026.05 | 45.44 | — | |
| IFDBackbone=Qwen2.5-7B2026.05 | 44.84 | — | |
| Data-WhispererBackbone=Qwen2.5-7B2026.05 | 44.56 | — | |
| IFDBackbone=Mistral-v0.2-7B2026.05 | 44.44 | — | |
| AlpagasusBackbone=Qwen2.5-7B2026.05 | 44.02 | — | |
| LongestBackbone=Qwen2.5-7B2026.05 | 43.92 | — | |
| RandomBackbone=Qwen2.5-7B2026.05 | 43.77 | — | |
| BaseBackbone=Qwen2.5-7B2026.05 | 43.67 | — | |
| DEITABackbone=Qwen2.5-7B2026.05 | 43.52 | — | |
| TMAPBackbone=Llama3-8B2026.05 | 42.99 | — | |
| AlpagasusBackbone=Llama3-8B2026.05 | 42.86 | — | |
| IFDBackbone=Llama3-8B2026.05 | 42.72 | — | |
| Data-WhispererBackbone=Llama3-8B2026.05 | 41.8 | — | |
| RandomBackbone=Llama3-8B2026.05 | 41.67 | — | |
| Full (100%)Backbone=Llama3-8B2026.05 | 40.21 | — | |
| DEITABackbone=Llama3-8B2026.05 | 38.89 | — | |
| BaseBackbone=Llama3-8B2026.05 | 37.96 | — | |
| LongestBackbone=Llama3-8B2026.05 | 37.3 | — | |
| CoSpaDiBase Model=Qwen 3 8B, Compression Ratio=0.22025.09 | — | 42.1 | |
| CoSpaDiBase Model=Qwen 3 8B, Compression Ratio=0.32025.09 | — | 38.4 | |
| CoSpaDiBase Model=Qwen 3 8B, Compression Ratio=0.42025.09 | — | 38.1 | |
| CoSpaDiBase Model=Qwen 3 14B, Compression Ratio=0.22025.09 | — | 40.7 | |
| CoSpaDiBase Model=Qwen 3 14B, Compression Ratio=0.42025.09 | — | 38 | |
| Qwen 3 14BCompression Ratio=Full2025.09 | — | 40.7 | |
| Qwen 3 8BCompression Ratio=Full2025.09 | — | 43.1 | |
| SVDLLMBase Model=Qwen 3 8B, Compression Ratio=0.22025.09 | — | 39.8 | |
| SVDLLMBase Model=Qwen 3 8B, Compression Ratio=0.32025.09 | — | 41.4 | |
| SVDLLMBase Model=Qwen 3 8B, Compression Ratio=0.42025.09 | — | 37.7 | |
| SVDLLMBase Model=Qwen 3 14B, Compression Ratio=0.22025.09 | — | 41.5 | |
| SVDLLMBase Model=Qwen 3 14B, Compression Ratio=0.42025.09 | — | 39.7 |