Logical Reasoning on ListOps
78AccuracyLoRA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LoRABase Model=Qwen 3 4B, Params (%)=0.41%, Rank=82026.02 | 78 | 0.9 | |
| SFTBase Model=Qwen 3 4B, Params (%)=100%2026.02 | 77.1 | 0 | |
| Post-block steeringBase Model=Qwen 3 4B, Params (%)=0.04%, Rank=82026.02 | 74 | -3.1 | |
| Post-block steeringBase Model=Llama-3.1-8B, Params (%)=0.02%, Rank=82026.02 | 69.1 | 1.9 | |
| ReFTBase Model=Qwen 3 4B, Params (%)=0.04%, Rank=82026.02 | 68.7 | -8.4 | |
| Ours r=1Model=Llama-3.1-8B, Params (%)=0.003%2026.02 | 68.5 | 1.3 | |
| LoRABase Model=Llama-3.1-8B, Params (%)=0.26%, Rank=82026.02 | 67.6 | 0.4 | |
| SFTBase Model=Llama-3.1-8B, Params (%)=100%2026.02 | 67.2 | 0 | |
| Ours (1-vec)Model=Llama-3.1-8B, Params (%)=0.001%2026.02 | 66.4 | -2.8 | |
| SFTBase Model=Llama-3.2-1B, Params (%)=100%2026.02 | 66.2 | 0 | |
| SFTBase Model=gemma-3-1b, Params (%)=100%2026.02 | 65.8 | 0 | |
| Post-block steeringBase Model=Llama-3.2-1B, Params (%)=0.04%, Rank=82026.02 | 65.4 | -0.8 | |
| LoFITModel=Qwen 3 4B, Params (%)=0.003%2026.02 | 65.3 | -11.8 | |
| Ours (1-vec)Model=Qwen 3 4B, Params (%)=0.003%2026.02 | 65.3 | -11.8 | |
| Post-block steeringBase Model=gemma-3-1b, Params (%)=0.04%, Rank=82026.02 | 65.2 | -0.6 | |
| Ours r=1Model=gemma-3-1b, Params (%)=0.005%2026.02 | 65.2 | -0.6 | |
| LoFITModel=Llama-3.1-8B, Params (%)=0.001%2026.02 | 64.8 | -2.4 | |
| Ours (1-vec)Model=Llama-3.2-1B, Params (%)=0.003%2026.02 | 64.6 | -1.6 | |
| JoLAModel=Qwen 3 4B, Params (%)=0.007%2026.02 | 64.5 | -12.6 | |
| ReFTBase Model=Llama-3.1-8B, Params (%)=0.02%, Rank=82026.02 | 64.4 | -2.8 | |
| Ours r=1Model=Llama-3.2-1B, Params (%)=0.005%2026.02 | 64.4 | -1.8 | |
| LoFITModel=Llama-3.2-1B, Params (%)=0.003%2026.02 | 64.1 | -2.1 | |
| JoLAModel=Llama-3.1-8B, Params (%)=0.003%2026.02 | 63.8 | -3.4 | |
| Ours r=1Model=Qwen 3 4B, Params (%)=0.005%2026.02 | 63.6 | -13.5 | |
| LoFITModel=gemma-3-1b, Params (%)=0.003%2026.02 | 62.3 | -3.5 | |
| JoLAModel=Llama-3.2-1B, Params (%)=0.007%2026.02 | 62.2 | -4 | |
| Ours (1-vec)Model=gemma-3-1b, Params (%)=0.003%2026.02 | 61.9 | -3.9 | |
| LoRABase Model=Llama-3.2-1B, Params (%)=0.45%, Rank=82026.02 | 60 | -6.2 | |
| JoLAModel=gemma-3-1b, Params (%)=0.007%2026.02 | 57.8 | -8 | |
| LoRABase Model=gemma-3-1b, Params (%)=0.45%, Rank=82026.02 | 53.6 | -12.2 | |
| ReFTBase Model=gemma-3-1b, Params (%)=0.04%, Rank=82026.02 | 50.3 | -15.5 | |
| ReFTBase Model=Llama-3.2-1B, Params (%)=0.04%, Rank=82026.02 | 49.3 | -16.9 |