Machine Translation on English-French (Accuracy)
75.8AccuracyState vector (inn.)
Evaluation Results
| Method | Links | |
|---|---|---|
| State vector (inn.)Model=Llama-2-13B, Evaluation Protocol=Few-shot2024.04 | 75.8 | |
| State vector (mom.)Model=Llama-2-13B, Evaluation Protocol=Few-shot2024.04 | 75.6 | |
| + LTVprompt=few-shot2025.02 | 75.4 | |
| Function vectorModel=Llama-2-13B, Evaluation Protocol=Few-shot2024.04 | 75.2 | |
| + SVprompt=few-shot2025.02 | 75 | |
| State vector (inn.)Model=Llama-2, Evaluation Protocol=Few-shot2024.04 | 74.6 | |
| ICL baselineModel=Llama-2-13B, Evaluation Protocol=Few-shot2024.04 | 74.5 | |
| + ICVprompt=few-shot2025.02 | 74.4 | |
| ICL baselineModel=Llama-2, Evaluation Protocol=Few-shot2024.04 | 74.3 | |
| State vector (mom.)Model=Llama-2, Evaluation Protocol=Few-shot2024.04 | 74.3 | |
| Task vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.04 | 73.8 | |
| Transformerprompt=few-shot2025.02 | 71.9 | |
| State vector (inn.)Model=GPT-J, Evaluation Protocol=Few-shot2024.04 | 70.9 | |
| Task vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.04 | 70.6 | |
| Task vectorModel=Llama-2-13B, Evaluation Protocol=Few-shot2024.04 | 70.5 | |
| State vector (mom.)Model=GPT-J, Evaluation Protocol=Few-shot2024.04 | 70.1 | |
| ICL baselineModel=GPT-J, Evaluation Protocol=Few-shot2024.04 | 69.9 | |
| + FVprompt=few-shot2025.02 | 68 | |
| State vector (mom.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 67.5 | |
| State vector (inn.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 66.5 | |
| Function vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.04 | 65.8 | |
| Function vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.04 | 65.2 | |
| Task vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 63.2 | |
| State vector (mom.)Model=Llama-2-13B, Evaluation Protocol=Zero-shot2024.04 | 55.9 | |
| + LTVprompt=zero-shot2025.02 | 53.5 | |
| Standard Promptingprompt=few-shot2025.02 | 52 | |
| State vector (inn.)Model=Llama-2-13B, Evaluation Protocol=Zero-shot2024.04 | 50.5 | |
| Task vectorModel=Llama-2-13B, Evaluation Protocol=Zero-shot2024.04 | 43.1 | |
| + FVprompt=zero-shot2025.02 | 40.2 | |
| State vector (mom.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 35.1 | |
| Task vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 32.2 | |
| State vector (inn.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 31.7 | |
| Function vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 29.1 | |
| + SVprompt=zero-shot2025.02 | 25.4 | |
| + ICVprompt=zero-shot2025.02 | 24.4 | |
| Transformerprompt=zero-shot2025.02 | 23.4 | |
| Function vectorModel=Llama-2-13B, Evaluation Protocol=Zero-shot2024.04 | 23.2 | |
| Function vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 21.6 | |
| RegularModel=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 7.2 | |
| RegularModel=Llama-2-13B, Evaluation Protocol=Zero-shot2024.04 | 0.2 | |
| RegularModel=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 0.1 | |
| Standard Promptingprompt=zero-shot2025.02 | 0 |