Word Relation Prediction on Person-Occupation
58.2AccuracyState vector (mom.)
Evaluation results
| Method | Links | |
|---|---|---|
| State vector (mom.)Model=Llama-2, Evaluation Protocol=Few-shot2024.04 | 58.2 | |
| State vector (inn.)Model=Llama-2, Evaluation Protocol=Few-shot2024.04 | 57 | |
| Task vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.04 | 56.4 | |
| ICL baselineModel=Llama-2, Evaluation Protocol=Few-shot2024.04 | 56.1 | |
| Function vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.04 | 54.2 | |
| State vector (mom.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 45.6 | |
| State vector (inn.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 42.7 | |
| State vector (mom.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 42.4 | |
| State vector (mom.)Model=GPT-J, Evaluation Protocol=Few-shot2024.04 | 30.4 | |
| Function vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.04 | 30.3 | |
| State vector (inn.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 30 | |
| State vector (inn.)Model=GPT-J, Evaluation Protocol=Few-shot2024.04 | 29.4 | |
| ICL baselineModel=GPT-J, Evaluation Protocol=Few-shot2024.04 | 29.3 | |
| Task vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 28.3 | |
| Task vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 27.9 | |
| Task vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.04 | 27.8 | |
| Function vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 11.1 | |
| Function vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 0.1 | |
| RegularModel=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 0 | |
| RegularModel=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 0 |