Knowledge Retrieval / Relation Prediction on Person-Instrument
0.786AccuracyState vector (mom.)
Evaluation Results
| Method | Links | |
|---|---|---|
| State vector (mom.)Model=Llama-2-13B, Evaluation Protocol=Few-shot2024.04 | 0.786 | |
| State vector (inn.)Model=Llama-2-13B, Evaluation Protocol=Few-shot2024.04 | 0.77 | |
| ICL baselineModel=Llama-2-13B, Evaluation Protocol=Few-shot2024.04 | 0.75 | |
| State vector (mom.)Model=Llama-2, Evaluation Protocol=Few-shot2024.04 | 0.749 | |
| Function vectorModel=Llama-2-13B, Evaluation Protocol=Few-shot2024.04 | 0.722 | |
| ICL baselineModel=Llama-2, Evaluation Protocol=Few-shot2024.04 | 0.717 | |
| Task vectorModel=Llama-2-13B, Evaluation Protocol=Few-shot2024.04 | 0.706 | |
| State vector (inn.)Model=Llama-2, Evaluation Protocol=Few-shot2024.04 | 0.701 | |
| State vector (mom.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 0.687 | |
| State vector (mom.)Model=Llama-2-13B, Evaluation Protocol=Zero-shot2024.04 | 0.685 | |
| State vector (inn.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 0.674 | |
| State vector (inn.)Model=Llama-2-13B, Evaluation Protocol=Zero-shot2024.04 | 0.666 | |
| Task vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.04 | 0.666 | |
| Task vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 0.618 | |
| Function vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.04 | 0.608 | |
| Task vectorModel=Llama-2-13B, Evaluation Protocol=Zero-shot2024.04 | 0.582 | |
| State vector (mom.)Model=GPT-J, Evaluation Protocol=Few-shot2024.04 | 0.519 | |
| State vector (mom.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 0.503 | |
| State vector (inn.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 0.493 | |
| Function vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.04 | 0.491 | |
| State vector (inn.)Model=GPT-J, Evaluation Protocol=Few-shot2024.04 | 0.465 | |
| ICL baselineModel=GPT-J, Evaluation Protocol=Few-shot2024.04 | 0.447 | |
| Task vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 0.444 | |
| Task vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.04 | 0.423 | |
| Function vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 0.113 | |
| Function vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 0.041 | |
| Function vectorModel=Llama-2-13B, Evaluation Protocol=Zero-shot2024.04 | 0.001 | |
| RegularModel=Llama-2-13B, Evaluation Protocol=Zero-shot2024.04 | 0 | |
| RegularModel=Llama-2, Evaluation Protocol=Zero-shot2024.04 | 0 | |
| RegularModel=GPT-J, Evaluation Protocol=Zero-shot2024.04 | 0 |