Causal Reasoning on CLadder 14 (original)
0.465NLLSAR (Shifting Attention to Relevance)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SAR (Shifting Attention to Relevance)Model=Gemma3-12B2026.01 | 0.465 | 13.32 | |
| Logits (Average)Model=Gemma3-12B2026.01 | 0.494 | 11.31 | |
| RCCModel=Gemma3-12B2026.01 | 0.53 | 11.9 | |
| RCCModel=Qwen3-8B2026.01 | 0.549 | 5.13 | |
| Logits (Average)Model=Qwen3-8B2026.01 | 0.552 | 12.57 | |
| UQAC (Attention Chain)Model=Gemma3-12B2026.01 | 0.933 | 10.1 | |
| SAR (Shifting Attention to Relevance)Model=Qwen3-8B2026.01 | 1.008 | 8.19 | |
| Self-EvaluationModel=Gemma3-12B2026.01 | 1.34 | 12.73 | |
| Self-ConsistencyModel=Gemma3-12B2026.01 | 2.109 | 15.28 | |
| UQAC (Attention Chain)Model=Qwen3-8B2026.01 | 2.271 | 6.36 | |
| Logits (Final)Model=Gemma3-12B2026.01 | 2.944 | 20.2 | |
| Self-ConsistencyModel=Qwen3-8B2026.01 | 3.763 | 11.42 | |
| Self-EvaluationModel=Qwen3-8B2026.01 | 4.957 | 17.59 | |
| Logits (Final)Model=Qwen3-8B2026.01 | 5.722 | 21.21 |