Function Vector Evaluation on SemEval Zero-Shot
65.86Avg AccuracyLLaMa-2 + vz Fine-tuned FV
Evaluation results
| Method | Links | |
|---|---|---|
| LLaMa-2 + vz Fine-tuned FVBackbone=LLaMa-2, Intervention=Fine-tuned FV2026.01 | 65.86 | |
| LLaMa-3.1 + vz Fine-tuned FVBackbone=LLaMa-3.1, Intervention=Fine-tuned FV2026.01 | 60.79 | |
| GPT-J + vz Fine-tuned FVBackbone=GPT-J, Intervention=Fine-tuned FV2026.01 | 60.78 | |
| GPT-2 + vz Fine-tuned FVBackbone=GPT-2, Intervention=Fine-tuned FV2026.01 | 46.42 | |
| LLaMa-2 + vt Initial FVBackbone=LLaMa-2, Intervention=Initial FV2026.01 | 42.82 | |
| LLaMa-3.1 + vt Initial FVBackbone=LLaMa-3.1, Intervention=Initial FV2026.01 | 32.72 | |
| GPT-J + vt Initial FVBackbone=GPT-J, Intervention=Initial FV2026.01 | 30.24 | |
| LLaMa-3.1Backbone=LLaMa-3.1, Intervention=Base2026.01 | 11.12 | |
| GPT-2 + vt Initial FVBackbone=GPT-2, Intervention=Initial FV2026.01 | 11.05 | |
| GPT-JBackbone=GPT-J, Intervention=Base2026.01 | 8.83 | |
| LLaMa-2Backbone=LLaMa-2, Intervention=Base2026.01 | 8.15 | |
| GPT-2Backbone=GPT-2, Intervention=Base2026.01 | 0.65 |