Analogy Solving on SAT Problems
58.8AccuracyvC
Evaluation results
| Method | Links | ||
|---|---|---|---|
| vCBackbone=LLaMa-3.1, Affine transformation=g() affine2026.01 | 58.8 | 0.093 | |
| vCBackbone=LLaMa-2, Affine transformation=g() affine2026.01 | 56.3 | 0.092 | |
| vCBackbone=LLaMa-3.1, Affine transformation=None2026.01 | 55.6 | 0.107 | |
| vCBackbone=LLaMa-2, Affine transformation=None2026.01 | 55 | 0.052 | |
| LLaMa-3.1Intervention=None2026.01 | 52.7 | — | |
| IFV-CFVBackbone=LLaMa-2, Affine transformation=g() affine2026.01 | 52.2 | 0.063 | |
| LLaMa-2Intervention=None2026.01 | 51.9 | — | |
| IFV-CFVBackbone=LLaMa-3.1, Affine transformation=g() affine2026.01 | 50.1 | 0.07 | |
| vCBackbone=GPT-J, Affine transformation=g() affine2026.01 | 49.1 | 0.062 | |
| IFV-CFVBackbone=LLaMa-3.1, Affine transformation=None2026.01 | 48.9 | 0.065 | |
| IFV-CFVBackbone=LLaMa-2, Affine transformation=None2026.01 | 46.9 | 0.027 | |
| vCBackbone=GPT-J, Affine transformation=None2026.01 | 44.4 | 0.028 | |
| IFV-CFVBackbone=GPT-J, Affine transformation=g() affine2026.01 | 36.7 | 0.045 | |
| IFV-CFVBackbone=GPT-J, Affine transformation=None2026.01 | 34.8 | 0.011 | |
| GPT-JIntervention=None2026.01 | 33.4 | — | |
| vCBackbone=GPT-2, Affine transformation=g() affine2026.01 | 28.6 | 0.083 | |
| vCBackbone=GPT-2, Affine transformation=None2026.01 | 26 | 0.035 | |
| IFV-CFVBackbone=GPT-2, Affine transformation=g() affine2026.01 | 23.2 | 0.03 | |
| IFV-CFVBackbone=GPT-2, Affine transformation=None2026.01 | 22.4 | 0.013 | |
| GPT-2Intervention=None2026.01 | 19.8 | — |