Function Vector Evaluation on Simple-Task Zero-Shot
89.98Average AccuracyLLaMa-3.1 + vz Fine-tuned FV
Evaluation results
| Method | Links | |
|---|---|---|
| LLaMa-3.1 + vz Fine-tuned FVBackbone=LLaMa-3.1, Intervention=Fine-tuned FV2026.01 | 89.98 | |
| LLaMa-2 + vz Fine-tuned FVBackbone=LLaMa-2, Intervention=Fine-tuned FV2026.01 | 88.84 | |
| GPT-J + vz Fine-tuned FVBackbone=GPT-J, Intervention=Fine-tuned FV2026.01 | 87.22 | |
| LLaMa-2 + vt Initial FVBackbone=LLaMa-2, Intervention=Initial FV2026.01 | 68.45 | |
| LLaMa-3.1 + vt Initial FVBackbone=LLaMa-3.1, Intervention=Initial FV2026.01 | 65.85 | |
| GPT-2 + vz Fine-tuned FVBackbone=GPT-2, Intervention=Fine-tuned FV2026.01 | 61.84 | |
| GPT-J + vt Initial FVBackbone=GPT-J, Intervention=Initial FV2026.01 | 54.11 | |
| GPT-2 + vt Initial FVBackbone=GPT-2, Intervention=Initial FV2026.01 | 16.68 | |
| LLaMa-2Backbone=LLaMa-2, Intervention=Base2026.01 | 13.28 | |
| GPT-JBackbone=GPT-J, Intervention=Base2026.01 | 5.13 | |
| LLaMa-3.1Backbone=LLaMa-3.1, Intervention=Base2026.01 | 2.75 | |
| GPT-2Backbone=GPT-2, Intervention=Base2026.01 | 1.39 |