List your company
ResearchFormerly WizWand

Semantic Antonym Prediction on Antonym

67AccuracyICL baseline
-2.6560815.4277133.511551.59529Apr 17, 2024Jun 5, 2024Jul 25, 2024Sep 12, 2024Nov 1, 2024Dec 20, 2024Feb 8, 2025
Updated 8mo ago

Evaluation results

MethodLinks
ICL baselineModel=Llama-2-13B, Evaluation Protocol=Few-shot2024.0467
State vector (inn.)Model=Llama-2, Evaluation Protocol=Few-shot2024.0466.2
State vector (mom.)Model=Llama-2-13B, Evaluation Protocol=Few-shot2024.0465.9
State vector (mom.)Model=Llama-2, Evaluation Protocol=Few-shot2024.0465.8
Function vectorModel=Llama-2-13B, Evaluation Protocol=Few-shot2024.0465.7
Task vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.0465.7
State vector (inn.)Model=Llama-2-13B, Evaluation Protocol=Few-shot2024.0465.5
Task vectorModel=Llama-2-13B, Evaluation Protocol=Few-shot2024.0464.8
ICL baselineModel=Llama-2, Evaluation Protocol=Few-shot2024.0464.8
State vector (inn.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.0461
State vector (mom.)Model=Llama-2, Evaluation Protocol=Zero-shot2024.0460.4
State vector (mom.)Model=GPT-J, Evaluation Protocol=Few-shot2024.0459.6
ICL baselineModel=GPT-J, Evaluation Protocol=Few-shot2024.0459.2
State vector (inn.)Model=GPT-J, Evaluation Protocol=Few-shot2024.0458.7
Task vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.0458.5
Function vectorModel=GPT-J, Evaluation Protocol=Few-shot2024.0456.4
Task vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.0456.2
Function vectorModel=Llama-2, Evaluation Protocol=Few-shot2024.0454.5
State vector (mom.)Model=Llama-2-13B, Evaluation Protocol=Zero-shot2024.0447.9
Function vectorModel=Llama-2-13B, Evaluation Protocol=Zero-shot2024.0447.1
State vector (inn.)Model=Llama-2-13B, Evaluation Protocol=Zero-shot2024.0447
Task vectorModel=Llama-2-13B, Evaluation Protocol=Zero-shot2024.0446
Function vectorModel=Llama-2, Evaluation Protocol=Zero-shot2024.0445.1
State vector (inn.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.0433.4
Function vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.0433.1
State vector (mom.)Model=GPT-J, Evaluation Protocol=Zero-shot2024.0431.1
Task vectorModel=GPT-J, Evaluation Protocol=Zero-shot2024.0423.6
RegularModel=GPT-J, Evaluation Protocol=Zero-shot2024.048.1
RegularModel=Llama-2-13B, Evaluation Protocol=Zero-shot2024.041.2
RegularModel=Llama-2, Evaluation Protocol=Zero-shot2024.041
+ LTVprompt=few-shot2025.020.656
+ SVprompt=few-shot2025.020.649
+ ICVprompt=few-shot2025.020.648
Transformerprompt=few-shot2025.020.645
+ FVprompt=few-shot2025.020.618
+ LTVprompt=zero-shot2025.020.535
+ FVprompt=zero-shot2025.020.464
Standard Promptingprompt=few-shot2025.020.375
Transformerprompt=zero-shot2025.020.352
+ ICVprompt=zero-shot2025.020.352
+ TVprompt=few-shot2025.020.345
+ SVprompt=zero-shot2025.020.336
+ TVprompt=zero-shot2025.020.317
Standard Promptingprompt=zero-shot2025.020.023