Commonsense Question Answering on CommonQA (accuracy, std, speedup)
84.4AccuracyTALE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TALEBackbone=Qwen 2.5 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=22025.10 | 84.4 | 0.17 | -6.6 | |
| BSBABackbone=Qwen 2.5 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=62025.10 | 80.5 | — | -19.9 | |
| BaselineBackbone=Qwen 2.5 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=02025.10 | 80.3 | — | — | |
| TALEBackbone=LLaMA 3.1 8B, Evaluation Protocol=0-shot, Layers Removed (#D)=32025.10 | 73.8 | 0.15 | -8.8 | |
| BSBABackbone=LLaMA 3.1 8B, Evaluation Protocol=0-shot, Layers Removed (#D)=62025.10 | 73.1 | — | -17.6 | |
| BaselineBackbone=LLaMA 3.1 8B, Evaluation Protocol=0-shot, Layers Removed (#D)=02025.10 | 72.9 | — | — | |
| TALEBackbone=Lucie 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=32025.10 | 68.6 | 0.27 | -9.1 | |
| TALEBackbone=Mistral 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=42025.10 | 64.4 | 0.19 | -12.3 | |
| BSBABackbone=Mistral 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=72025.10 | 61.6 | — | -21.5 | |
| BaselineBackbone=Mistral 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=02025.10 | 61 | — | — | |
| BSBABackbone=Lucie 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=172025.10 | 54.6 | — | -48.2 | |
| BaselineBackbone=Lucie 7B, Evaluation Protocol=0-shot, Layers Removed (#D)=02025.10 | 54.2 | — | — |