Word Relation Prediction on Product-Company
82.8AccuracyState vector (inn.)
Evaluation results
| Method | Links | |
|---|---|---|
| State vector (inn.)Model=Llama-2, Evaluation Protocol=Few-shot2024.04 | 82.8 | |
| State vector (mom.)Model=Llama-2, Evaluation Protocol=Few-shot2024.04 | 82 | |
| Task vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.04 | 81.9 | |
| ICL baselineModel=Llama-2, Evaluation Protocol=Few-shot2024.04 | 80.8 | |
| Function vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.04 | 76 | |
| State vector (mom.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 71.3 | |
| State vector (inn.)Model=GPT-J, Evaluation Protocol=Few-shot2024.04 | 66.3 | |
| Task vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.04 | 66 | |
| State vector (inn.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 64.5 | |
| State vector (mom.)Model=GPT-J, Evaluation Protocol=Few-shot2024.04 | 63.8 | |
| ICL baselineModel=GPT-J, Evaluation Protocol=Few-shot2024.04 | 62.5 | |
| Function vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.04 | 58.5 | |
| Task vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 55.5 | |
| Function vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 46.3 | |
| State vector (mom.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 44.2 | |
| Task vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 43.8 | |
| State vector (inn.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 42.8 | |
| Function vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 25.6 | |
| RegularModel=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 1.9 | |
| RegularModel=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 0.4 |