Semantic Antonym Prediction on Antonym
67AccuracyICL baseline
Evaluation Results
| Method | Links | |
|---|---|---|
| ICL baselineModel=Llama-2-13B, Evaluation Protocol=Few-shot2024.04 | 67 | |
| State vector (inn.)Model=Llama-2, Evaluation Protocol=Few-shot2024.04 | 66.2 | |
| State vector (mom.)Model=Llama-2-13B, Evaluation Protocol=Few-shot2024.04 | 65.9 | |
| State vector (mom.)Model=Llama-2, Evaluation Protocol=Few-shot2024.04 | 65.8 | |
| Function vectorModel=Llama-2-13B, Evaluation Protocol=Few-shot2024.04 | 65.7 | |
| Task vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.04 | 65.7 | |
| State vector (inn.)Model=Llama-2-13B, Evaluation Protocol=Few-shot2024.04 | 65.5 | |
| Task vectorModel=Llama-2-13B, Evaluation Protocol=Few-shot2024.04 | 64.8 | |
| ICL baselineModel=Llama-2, Evaluation Protocol=Few-shot2024.04 | 64.8 | |
| State vector (inn.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 61 | |
| State vector (mom.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 60.4 | |
| State vector (mom.)Model=GPT-J, Evaluation Protocol=Few-shot2024.04 | 59.6 | |
| ICL baselineModel=GPT-J, Evaluation Protocol=Few-shot2024.04 | 59.2 | |
| State vector (inn.)Model=GPT-J, Evaluation Protocol=Few-shot2024.04 | 58.7 | |
| Task vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.04 | 58.5 | |
| Function vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.04 | 56.4 | |
| Task vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 56.2 | |
| Function vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.04 | 54.5 | |
| State vector (mom.)Model=Llama-2-13B, Evaluation Protocol=Zero-shot2024.04 | 47.9 | |
| Function vectorModel=Llama-2-13B, Evaluation Protocol=Zero-shot2024.04 | 47.1 | |
| State vector (inn.)Model=Llama-2-13B, Evaluation Protocol=Zero-shot2024.04 | 47 | |
| Task vectorModel=Llama-2-13B, Evaluation Protocol=Zero-shot2024.04 | 46 | |
| Function vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 45.1 | |
| State vector (inn.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 33.4 | |
| Function vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 33.1 | |
| State vector (mom.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 31.1 | |
| Task vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 23.6 | |
| RegularModel=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 8.1 | |
| RegularModel=Llama-2-13B, Evaluation Protocol=Zero-shot2024.04 | 1.2 | |
| RegularModel=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 1 | |
| + LTVprompt=few-shot2025.02 | 0.656 | |
| + SVprompt=few-shot2025.02 | 0.649 | |
| + ICVprompt=few-shot2025.02 | 0.648 | |
| Transformerprompt=few-shot2025.02 | 0.645 | |
| + FVprompt=few-shot2025.02 | 0.618 | |
| + LTVprompt=zero-shot2025.02 | 0.535 | |
| + FVprompt=zero-shot2025.02 | 0.464 | |
| Standard Promptingprompt=few-shot2025.02 | 0.375 | |
| Transformerprompt=zero-shot2025.02 | 0.352 | |
| + ICVprompt=zero-shot2025.02 | 0.352 | |
| + TVprompt=few-shot2025.02 | 0.345 | |
| + SVprompt=zero-shot2025.02 | 0.336 | |
| + TVprompt=zero-shot2025.02 | 0.317 | |
| Standard Promptingprompt=zero-shot2025.02 | 0.023 |