List your company
ResearchFormerly WizWand

Knowledge Retrieval / Relation Prediction on Person-Instrument

0.786AccuracyState vector (mom.)
-0.031440.180780.3930.60522Apr 17, 2024
Updated 8mo ago

Evaluation results

MethodLinks
State vector (mom.)Model=Llama-2-13B, Evaluation Protocol=Few-shot2024.040.786
State vector (inn.)Model=Llama-2-13B, Evaluation Protocol=Few-shot2024.040.77
ICL baselineModel=Llama-2-13B, Evaluation Protocol=Few-shot2024.040.75
State vector (mom.)Model=Llama-2, Evaluation Protocol=Few-shot2024.040.749
Function vectorModel=Llama-2-13B, Evaluation Protocol=Few-shot2024.040.722
ICL baselineModel=Llama-2, Evaluation Protocol=Few-shot2024.040.717
Task vectorModel=Llama-2-13B, Evaluation Protocol=Few-shot2024.040.706
State vector (inn.)Model=Llama-2, Evaluation Protocol=Few-shot2024.040.701
State vector (mom.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.040.687
State vector (mom.)Model=Llama-2-13B, Evaluation Protocol=Zero-shot2024.040.685
State vector (inn.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.040.674
State vector (inn.)Model=Llama-2-13B, Evaluation Protocol=Zero-shot2024.040.666
Task vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.040.666
Task vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.040.618
Function vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.040.608
Task vectorModel=Llama-2-13B, Evaluation Protocol=Zero-shot2024.040.582
State vector (mom.)Model=GPT-J, Evaluation Protocol=Few-shot2024.040.519
State vector (mom.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.040.503
State vector (inn.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.040.493
Function vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.040.491
State vector (inn.)Model=GPT-J, Evaluation Protocol=Few-shot2024.040.465
ICL baselineModel=GPT-J, Evaluation Protocol=Few-shot2024.040.447
Task vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.040.444
Task vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.040.423
Function vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.040.113
Function vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.040.041
Function vectorModel=Llama-2-13B, Evaluation Protocol=Zero-shot2024.040.001
RegularModel=Llama-2-13B, Evaluation Protocol=Zero-shot2024.040
RegularModel=Llama-2, Evaluation Protocol=Zero-shot2024.040
RegularModel=GPT-J, Evaluation Protocol=Zero-shot2024.040