List your company
ResearchFormerly WizWand

Word Relation Prediction on Landmark-Continent

87.6AccuracyState vector (mom.)
-3.50420.14843.867.452Apr 17, 2024
Updated 8mo ago

Evaluation results

MethodLinks
State vector (mom.)Model=Llama-2, Evaluation Protocol=Few-shot2024.0487.6
State vector (inn.)Model=Llama-2, Evaluation Protocol=Few-shot2024.0487.5
ICL baselineModel=Llama-2, Evaluation Protocol=Few-shot2024.0487
Task vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.0486.7
Function vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.0484.2
State vector (inn.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.0481
State vector (mom.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.0477.7
ICL baselineModel=GPT-J, Evaluation Protocol=Few-shot2024.0469.3
Function vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.0469.2
State vector (mom.)Model=GPT-J, Evaluation Protocol=Few-shot2024.0468.6
State vector (inn.)Model=GPT-J, Evaluation Protocol=Few-shot2024.0466.4
Task vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.0463.1
State vector (inn.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.0461.9
State vector (mom.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.0460.3
Task vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.0457.8
Task vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.0441.3
Function vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.0432.9
Function vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.0422.5
RegularModel=GPT-J, Evaluation Protocol=Zero-shot2024.040.9
RegularModel=Llama-2, Evaluation Protocol=Zero-shot2024.040