Relational Similarity on Relational Similarity Human Judgments Exp. 1
0.864Pearson Correlation (r)Fine-tuned FV
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Fine-tuned FVModel=LLaMa-3.1, Intervention Protocol=vz Fine-tuned FV2026.01 | 0.864 | 0.84 | |
| Fine-tuned FVModel=LLaMa-2, Intervention Protocol=vz Fine-tuned FV2026.01 | 0.846 | 0.82 | |
| Initial FVModel=LLaMa-2, Intervention Protocol=vt Initial FV2026.01 | 0.816 | 0.8 | |
| Initial FVModel=LLaMa-3.1, Intervention Protocol=vt Initial FV2026.01 | 0.811 | 0.78 | |
| Fine-tuned FVModel=GPT-2, Intervention Protocol=vz Fine-tuned FV2026.01 | 0.807 | 0.78 | |
| Fine-tuned FVModel=GPT-J, Intervention Protocol=vz Fine-tuned FV2026.01 | 0.788 | 0.76 | |
| Initial FVModel=GPT-J, Intervention Protocol=vt Initial FV2026.01 | 0.779 | 0.75 | |
| Initial FVModel=GPT-2, Intervention Protocol=vt Initial FV2026.01 | 0.778 | 0.75 | |
| LLaMa-2Intervention Protocol=Baseline2026.01 | 0.693 | 0.65 | |
| LLaMa-3.1Intervention Protocol=Baseline2026.01 | 0.67 | 0.63 | |
| GPT-2Intervention Protocol=Baseline2026.01 | 0.574 | 0.52 | |
| GPT-JIntervention Protocol=Baseline2026.01 | 0.423 | 0.36 |