Mathematical Reasoning on Math500 (Accuracy, Std, Speedup)
38.2AccuracyTALE
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| TALEBackbone=Qwen 2.5 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=22025.10 | 38.2 | 0.22 | -6.6 | |
| BSBABackbone=Qwen 2.5 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=42025.10 | 32.1 | — | -13.3 | |
| BaselineBackbone=Qwen 2.5 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=02025.10 | 31 | — | — | |
| TALEBackbone=LLaMA 3.1 8B, Evaluation Protocol=0-shot, Layers Removed (#D)=22025.10 | 28.2 | 0.18 | -6 | |
| TALEBackbone=Mistral 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=12025.10 | 28 | 0.23 | -3.1 | |
| BSBABackbone=LLaMA 3.1 8B, Evaluation Protocol=0-shot, Layers Removed (#D)=32025.10 | 27.4 | — | -9.1 | |
| TALEBackbone=Lucie 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=22025.10 | 26.1 | 0.2 | -6 | |
| BaselineBackbone=LLaMA 3.1 8B, Evaluation Protocol=0-shot, Layers Removed (#D)=02025.10 | 25.4 | — | — | |
| BSBABackbone=Lucie 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=32025.10 | 25.1 | — | -9.1 | |
| BaselineBackbone=Lucie 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=02025.10 | 21 | — | — | |
| BaselineBackbone=Mistral 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=02025.10 | 19 | — | — | |
| BSBABackbone=Mistral 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=42025.10 | 18.8 | — | -12.3 |