Causal Reasoning on CLadder ID (held-out)
98P0 ScoreMistral-7B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Mistral-7BAdapter type=Vernier, recipe=default, lr=1e-42026.06 | 98 | 99.5 | -1.5 | |
| Qwen2.5-32BAdapter type=Vernier, QLoRA=true2026.06 | 97 | 96.5 | 0.5 | |
| Qwen2.5-14BAdapter type=Vernier, QLoRA=true2026.06 | 95.5 | 95 | 0.5 | |
| Llama-3.1-8BAdapter type=Vernier2026.06 | 94.5 | 94.8 | -0.3 | |
| Qwen2.5-7BAdapter type=Vernier2026.06 | 86.8 | 89.3 | -2.5 | |
| Qwen2.5-7BAdapter type=Base2026.06 | 62.5 | 54.5 | 8 | |
| Qwen2.5-14BAdapter type=Base, QLoRA=true2026.06 | 61.5 | 58 | 3.5 | |
| Mistral-7BAdapter type=Base2026.06 | 61 | 56.5 | 4.5 | |
| Mistral-7BAdapter type=Vernier, recipe=rescued, lr=5e-6, early stopping step=1502026.06 | 60.5 | 60 | 0.5 | |
| Phi-3-mini (3.8B)Adapter type=Base2026.06 | 58.5 | 54 | 4.5 | |
| Qwen2.5-32BAdapter type=Base, QLoRA=true2026.06 | 56.5 | 58.5 | -2 | |
| Phi-3-mini (3.8B)Adapter type=Vernier2026.06 | 55 | 55 | 0 | |
| Llama-3.1-8BAdapter type=Base2026.06 | 54 | 51.5 | 2.5 |