Word-in-Context Classification on WiC
70.1AccuracyFT
Evaluation results
| Method | Links | |
|---|---|---|
| FTOptimization Algorithm=Adam, Tuning Strategy=Full Tuning, Model=OPT-13B2026.01 | 70.1 | |
| Hybrid-LoRAModel=Vicuna-7b-v1.52026.04 | 66.8 | |
| Hybrid-PromptModel=Vicuna-7b-v1.52026.04 | 64.7 | |
| Hybrid-PromptModel=Llama-2-7b2026.04 | 64.4 | |
| SubZero-GV (Prefix)Optimization Algorithm=SubZero, Tuning Strategy=Prefix Tuning, Guiding Vectors=true, Model=OPT-13B2026.01 | 64.1 | |
| SubZero-GV (LoRA)Optimization Algorithm=SubZero, Tuning Strategy=LoRA, Guiding Vectors=true, Model=OPT-13B2026.01 | 63.9 | |
| Hybrid-PromptModel=OPT-1.3b2026.04 | 63.3 | |
| SubZero-GV (FT)Optimization Algorithm=SubZero, Tuning Strategy=Full Tuning, Guiding Vectors=true, Model=OPT-13B2026.01 | 62.7 | |
| MeZO-GV (Prefix)Optimization Algorithm=MeZO, Tuning Strategy=Prefix Tuning, Guiding Vectors=true, Model=OPT-13B2026.01 | 61.8 | |
| Hybrid-LoRAModel=Llama-2-7b2026.04 | 61.7 | |
| FO-LoRAModel=Llama-2-7b2026.04 | 61.6 | |
| MeZO-GV (FT)Optimization Algorithm=MeZO, Tuning Strategy=Full Tuning, Guiding Vectors=true, Model=OPT-13B2026.01 | 61.4 | |
| MeZO (FT)Optimization Algorithm=MeZO, Tuning Strategy=Full Tuning, Guiding Vectors=false, Model=OPT-13B2026.01 | 61.1 | |
| SubZero (Prefix)Optimization Algorithm=SubZero, Tuning Strategy=Prefix Tuning, Guiding Vectors=false, Model=OPT-13B2026.01 | 61.1 | |
| FO-PromptModel=Vicuna-7b-v1.52026.04 | 61 | |
| MeZO-GV (LoRA)Optimization Algorithm=MeZO, Tuning Strategy=LoRA, Guiding Vectors=true, Model=OPT-13B2026.01 | 60.9 | |
| SubZero (FT)Optimization Algorithm=SubZero, Tuning Strategy=Full Tuning, Guiding Vectors=false, Model=OPT-13B2026.01 | 60.8 | |
| MeZO-GV(LoRA)Backbone=OPT-1.3B, Optimization Method=MeZO, Adaptation Technique=LoRA, Guiding Vectors (GV)=true2026.01 | 60.7 | |
| ZO-AdaMU (2x)Optimization Algorithm=ZO-AdaMU, Tuning Strategy=Full Tuning, Training Steps=2x, Model=OPT-13B2026.01 | 60.7 | |
| SubZero (LoRA)Optimization Algorithm=SubZero, Tuning Strategy=LoRA, Guiding Vectors=false, Model=OPT-13B2026.01 | 60.3 | |
| MeZO(LoRA)Backbone=OPT-1.3B, Optimization Method=MeZO, Adaptation Technique=LoRA2026.01 | 60.2 | |
| HiZOO (LoRA)Optimization Algorithm=HiZOO, Tuning Strategy=LoRA, Model=OPT-13B2026.01 | 60.2 | |
| HiZOO (Prefix)Optimization Algorithm=HiZOO, Tuning Strategy=Prefix Tuning, Model=OPT-13B2026.01 | 60 | |
| MeZO (Prefix)Optimization Algorithm=MeZO, Tuning Strategy=Prefix Tuning, Guiding Vectors=false, Model=OPT-13B2026.01 | 59.9 | |
| MeZO (LoRA)Optimization Algorithm=MeZO, Tuning Strategy=LoRA, Guiding Vectors=false, Model=OPT-13B2026.01 | 59.7 | |
| HiZOOOptimization Algorithm=HiZOO, Tuning Strategy=Full Tuning, Model=OPT-13B2026.01 | 59.4 | |
| LessLexExternal Lexical Resources=true2021.10 | 59.2 | |
| DeConfExternal Lexical Resources=true2021.10 | 58.7 | |
| FO-PromptModel=Llama-2-7b2026.04 | 58.5 | |
| FO-LoRAModel=Vicuna-7b-v1.52026.04 | 58.5 | |
| Sense-aware EmbeddingsExternal Lexical Resources=false2021.10 | 58.3 | |
| MeZO-GV(Prefix)Backbone=OPT-1.3B, Optimization Method=MeZO, Adaptation Technique=Prefix Tuning, Guiding Vectors (GV)=true2026.01 | 58.2 | |
| ZO-AdaMU (Prefix)Optimization Algorithm=ZO-AdaMU, Tuning Strategy=Prefix Tuning, Model=OPT-13B2026.01 | 58.2 | |
| SW2VExternal Lexical Resources=true2021.10 | 58.1 | |
| MeZO-GV(FT)Backbone=OPT-1.3B, Optimization Method=MeZO, Adaptation Technique=Fine-tuning, Guiding Vectors (GV)=true2026.01 | 58 | |
| FO-PromptModel=OPT-1.3b2026.04 | 57.5 | |
| MeZO(FT)Backbone=OPT-1.3B, Optimization Method=MeZO, Adaptation Technique=Fine-tuning2026.01 | 57.2 | |
| FO-PrefixModel=Vicuna-7b-v1.52026.04 | 56.6 | |
| ZO-AdaMU (LoRA)Optimization Algorithm=ZO-AdaMU, Tuning Strategy=LoRA, Model=OPT-13B2026.01 | 56.4 | |
| MeZO(Prefix)Backbone=OPT-1.3B, Optimization Method=MeZO, Adaptation Technique=Prefix Tuning2026.01 | 56 | |
| Hybrid-LoRAModel=OPT-1.3b2026.04 | 55.5 | |
| Zero-shotEvaluation Protocol=Zero-shot, Model=OPT-13B2026.01 | 55 | |
| JBTExternal Lexical Resources=false2021.10 | 53.6 | |
| Zero-shotBackbone=OPT-1.3B, Learning Strategy=Zero-shot2026.01 | 53.5 | |
| Hybrid-PrefixModel=OPT-1.3b2026.04 | 52.7 | |
| FO-PrefixModel=OPT-1.3b2026.04 | 52.5 | |
| FO-LoRAModel=OPT-1.3b2026.04 | 52.5 | |
| Hybrid-PrefixModel=Vicuna-7b-v1.52026.04 | 52.4 | |
| FO-PrefixModel=Llama-2-7b2026.04 | 51.7 | |
| Hybrid-PrefixModel=Llama-2-7b2026.04 | 51.5 | |
| ICLEvaluation Protocol=In-context Learning, Model=OPT-13B2026.01 | 50.5 | |
| ICLBackbone=OPT-1.3B, Learning Strategy=In-context Learning2026.01 | 50.3 |