Classification on SST-2 (Accuracy)
96.3AccuracyFull Fine-Tuning (FT)
Evaluation results
| Method | Links | |
|---|---|---|
| Full Fine-Tuning (FT)Model=Qwen2.5-14B, Trainable Parameters=14.7B2025.12 | 96.3 | |
| LoRAModel=LLAMA3.1-8B, Trainable Parameters=3.4M2025.12 | 96.2 | |
| Partial-LoRAModel=LLAMA3.1-8B, Trainable Parameters=80k2025.12 | 96.2 | |
| Partial-LoRAModel=Qwen2.5-14B, Trainable Parameters=92K2025.12 | 96.2 | |
| Masking (0.001%)Model=Qwen2.5-14B, Trainable Parameters=149K2025.12 | 96.1 | |
| Full Fine-Tuning (FT)Model=LLAMA3.1-8B, Trainable Parameters=8B2025.12 | 96 | |
| Hybrid-PromptModel=Llama-2-7b2026.04 | 95.9 | |
| First Order Adamw FTModel Backbone=Llama2-7B, Fine-tuning Strategy=Full Fine-tuning, Optimization Algorithm=First Order Adamw, Number of training examples=10002026.06 | 95.87 | |
| LoRAModel=Qwen2.5-14B, Trainable Parameters=6.3M2025.12 | 95.8 | |
| FO-PromptModel=Llama-2-7b2026.04 | 95.6 | |
| Masking (0.001%)Model=LLAMA2-7B, Trainable Parameters=68K2025.12 | 95.5 | |
| LoRAModel=LLAMA2-7B, Trainable Parameters=4.2M2025.12 | 95.4 | |
| Fine-tuningModel Backbone=OPT-6.7B, Model Precision=16 bits, Memory Profiling=26.8GB2025.05 | 95.4 | |
| Masking (0.001%)Model=LLAMA3.1-8B, Trainable Parameters=80K2025.12 | 95.3 | |
| Hybrid-PromptModel=Vicuna-7b-v1.52026.04 | 95 | |
| Partial-LoRAModel=LLAMA2-7B, Trainable Parameters=64K2025.12 | 94.9 | |
| ICLModel=LLaMA-2-13B2026.05 | 94.8 | |
| SubZero-GV (FT)Optimization Algorithm=SubZero, Tuning Strategy=Full Tuning, Guiding Vectors=true, Model=OPT-13B2026.01 | 94.7 | |
| Full Fine-Tuning (FT)Model=LLAMA2-7B, Trainable Parameters=6.7B2025.12 | 94.7 | |
| FO-LoRAModel=Llama-2-7b2026.04 | 94.6 | |
| FO-LoRAModel=Vicuna-7b-v1.52026.04 | 94.6 | |
| FO-PromptModel=Vicuna-7b-v1.52026.04 | 94.4 | |
| ICLModel=LLaMA-3.1-8B2026.05 | 94.2 | |
| SubZero-GV (LoRA)Optimization Algorithm=SubZero, Tuning Strategy=LoRA, Guiding Vectors=true, Model=OPT-13B2026.01 | 94 | |
| MeZO-GV (FT)Optimization Algorithm=MeZO, Tuning Strategy=Full Tuning, Guiding Vectors=true, Model=OPT-13B2026.01 | 93.9 | |
| SubZero (LoRA)Optimization Algorithm=SubZero, Tuning Strategy=LoRA, Guiding Vectors=false, Model=OPT-13B2026.01 | 93.8 | |
| Fine-tuningModel Backbone=Llama-3-8B, Model Precision=16 bits, Memory Profiling=31.9GB2025.05 | 93.7 | |
| MeZO-GV(LoRA)Backbone=OPT-1.3B, Optimization Method=MeZO, Adaptation Technique=LoRA, Guiding Vectors (GV)=true2026.01 | 93.5 | |
| Hybrid-LoRAModel=Llama-2-7b2026.04 | 93.4 | |
| SubZero-GV (Prefix)Optimization Algorithm=SubZero, Tuning Strategy=Prefix Tuning, Guiding Vectors=true, Model=OPT-13B2026.01 | 93.1 | |
| MeZO-GV(FT)Backbone=OPT-1.3B, Optimization Method=MeZO, Adaptation Technique=Fine-tuning, Guiding Vectors (GV)=true2026.01 | 93 | |
| MeZOModel Backbone=OPT-6.7B, Model Precision=16 bits, Memory Profiling=14.8GB2025.05 | 93 | |
| QZOModel Backbone=Llama-3-8B, Model Precision=4 bits, Memory Profiling=6.3GB2025.05 | 93 | |
| Fine-tuningModel Backbone=Llama-2-7B, Model Precision=16 bits, Memory Profiling=26.0GB2025.05 | 92.8 | |
| MeZO LoRAModel Backbone=Llama2-7B, Fine-tuning Strategy=LoRA, Optimization Algorithm=MeZO, Number of training examples=10002026.06 | 92.66 | |
| MeZOModel Backbone=Llama-3-8B, Model Precision=16 bits, Memory Profiling=20.5GB2025.05 | 92.5 | |
| MeZO-GV (Prefix)Optimization Algorithm=MeZO, Tuning Strategy=Prefix Tuning, Guiding Vectors=true, Model=OPT-13B2026.01 | 92.4 | |
| MeZO FTModel Backbone=Llama2-7B, Fine-tuning Strategy=Full Fine-tuning, Optimization Algorithm=MeZO, Number of training examples=10002026.06 | 92.32 | |
| Hybrid-LoRAModel=OPT-1.3b2026.04 | 92.3 | |
| Hybrid-LoRAModel=Vicuna-7b-v1.52026.04 | 92.2 | |
| FO-PrefixModel=OPT-1.3b2026.04 | 92.2 | |
| FO-LoRAModel=OPT-1.3b2026.04 | 92.2 | |
| MeZO-GV(Prefix)Backbone=OPT-1.3B, Optimization Method=MeZO, Adaptation Technique=Prefix Tuning, Guiding Vectors (GV)=true2026.01 | 92.1 | |
| ZO-AdaMU (2x)Optimization Algorithm=ZO-AdaMU, Tuning Strategy=Full Tuning, Training Steps=2x, Model=OPT-13B2026.01 | 92.1 | |
| HiZOOOptimization Algorithm=HiZOO, Tuning Strategy=Full Tuning, Model=OPT-13B2026.01 | 92.1 | |
| SubZero (FT)Optimization Algorithm=SubZero, Tuning Strategy=Full Tuning, Guiding Vectors=false, Model=OPT-13B2026.01 | 92.1 | |
| HiZOO (Prefix)Optimization Algorithm=HiZOO, Tuning Strategy=Prefix Tuning, Model=OPT-13B2026.01 | 92 | |
| FTOptimization Algorithm=Adam, Tuning Strategy=Full Tuning, Model=OPT-13B2026.01 | 92 | |
| SubZero (Prefix)Optimization Algorithm=SubZero, Tuning Strategy=Prefix Tuning, Guiding Vectors=false, Model=OPT-13B2026.01 | 91.7 | |
| Hybrid-PromptModel=OPT-1.3b2026.04 | 91.7 | |
| Hybrid-PrefixModel=OPT-1.3b2026.04 | 91.7 | |
| MeZO-GV (LoRA)Optimization Algorithm=MeZO, Tuning Strategy=LoRA, Guiding Vectors=true, Model=OPT-13B2026.01 | 91.6 | |
| Hybrid-PrefixModel=Llama-2-7b2026.04 | 91.6 | |
| MeZO (FT)Optimization Algorithm=MeZO, Tuning Strategy=Full Tuning, Guiding Vectors=false, Model=OPT-13B2026.01 | 91.4 | |
| FO-PromptModel=OPT-1.3b2026.04 | 91.3 | |
| FO-PrefixModel=Llama-2-7b2026.04 | 91.1 | |
| Dominant-layer ZO LoRAModel Backbone=Llama2-7B, Fine-tuning Strategy=LoRA, Optimization Algorithm=Dominant-layer ZO, Number of training examples=10002026.06 | 91.02 | |
| MeZO(LoRA)Backbone=OPT-1.3B, Optimization Method=MeZO, Adaptation Technique=LoRA2026.01 | 90.8 | |
| I2CLModel=LLaMA-3.1-8B2026.05 | 90.8 | |
| Dominant-layer ZO FTModel Backbone=Llama2-7B, Fine-tuning Strategy=Full Fine-tuning, Optimization Algorithm=Dominant-layer ZO, Number of training examples=10002026.06 | 90.79 | |
| MeZO (Prefix)Optimization Algorithm=MeZO, Tuning Strategy=Prefix Tuning, Guiding Vectors=false, Model=OPT-13B2026.01 | 90.7 | |
| Hybrid-PrefixModel=Vicuna-7b-v1.52026.04 | 90.7 | |
| HiZOO (LoRA)Optimization Algorithm=HiZOO, Tuning Strategy=LoRA, Model=OPT-13B2026.01 | 90.6 | |
| MeZO(Prefix)Backbone=OPT-1.3B, Optimization Method=MeZO, Adaptation Technique=Prefix Tuning2026.01 | 90.1 | |
| LTVModel=LLaMA-3.1-8B2026.05 | 90.08 | |
| QZOModel Backbone=Llama-2-7B, Model Precision=4 bits, Memory Profiling=5.0GB2025.05 | 90 | |
| FO-PrefixModel=Vicuna-7b-v1.52026.04 | 90 | |
| MeZO (LoRA)Optimization Algorithm=MeZO, Tuning Strategy=LoRA, Guiding Vectors=false, Model=OPT-13B2026.01 | 89.6 | |
| MeZO(FT)Backbone=OPT-1.3B, Optimization Method=MeZO, Adaptation Technique=Fine-tuning2026.01 | 89.2 | |
| LTVModel=LLaMA-2-13B2026.05 | 88.96 | |
| ZO-AdaMU (LoRA)Optimization Algorithm=ZO-AdaMU, Tuning Strategy=LoRA, Model=OPT-13B2026.01 | 88 | |
| ZO-AdaMU (Prefix)Optimization Algorithm=ZO-AdaMU, Tuning Strategy=Prefix Tuning, Model=OPT-13B2026.01 | 88 | |
| Task VectorModel=LLaMA-3.1-8B2026.05 | 87.88 | |
| QZOModel Backbone=OPT-6.7B, Model Precision=4 bits, Memory Profiling=4.8GB2025.05 | 87.6 | |
| Function VectorModel=LLaMA-3.1-8B2026.05 | 87.32 | |
| State VectorModel=LLaMA-2-13B2026.05 | 87.2 | |
| ICLEvaluation Protocol=In-context Learning, Model=OPT-13B2026.01 | 87 | |
| Zero-shotModel=LLaMA-3.1-8B2026.05 | 86.8 | |
| Llama-3-8BBackbone=Llama-3-8B2025.05 | 84.26 | |
| State VectorModel=LLaMA-3.1-8B2026.05 | 84.12 | |
| MeZOModel Backbone=Llama-2-7B, Model Precision=16 bits, Memory Profiling=14.8GB2025.05 | 83.5 | |
| MeZOBackbone=Llama-2-7B, Precision=BF162026.05 | 83.5 | |
| ASTRABackbone=Llama-3-8B, Group Size (G)=322025.05 | 83.14 | |
| Task VectorModel=LLaMA-2-13B2026.05 | 82.8 | |
| ASTRABackbone=Llama-3-8B, Group Size (G)=162025.05 | 81.79 | |
| QZOModel=Llama-2-13B, Precision=2 bits, Memory Profiling=5.78GB, Evaluation Protocol=Zeroth-order optimization2025.05 | 80.5 | |
| MeZOBackbone=Qwen-2.5-1.5B, Precision=BF162026.05 | 80.2 | |
| ICLBackbone=OPT-1.3B, Learning Strategy=In-context Learning2026.01 | 80 | |
| CAQ-ZOBackbone=Llama-2-7B, Precision=NF42026.05 | 76.5 | |
| ASTRABackbone=Llama-3-8B, Group Size (G)=12025.05 | 75.45 | |
| Function VectorModel=LLaMA-2-13B2026.05 | 72.44 | |
| QuZOBackbone=Llama-2-7B, Precision=NF42026.05 | 72.1 | |
| CAQ-ZOBackbone=Qwen-2.5-1.5B, Precision=NF42026.05 | 71.3 | |
| Zero-shotModel=LLaMA-2-13B2026.05 | 66.6 | |
| I2CLModel=LLaMA-2-13B2026.05 | 64.8 | |
| Zero-ShotModel Backbone=OPT-6.7B, Model Precision=16 bits2025.05 | 61.2 | |
| Zero-Shot-QModel Backbone=OPT-6.7B, Model Precision=4 bits2025.05 | 60.1 | |
| Zero-Shot-QBackbone=Qwen-2.5-1.5B, Precision=NF42026.05 | 60.1 | |
| Zero-ShotModel Backbone=Llama-3-8B, Model Precision=16 bits2025.05 | 59.6 | |
| Zero-shotEvaluation Protocol=Zero-shot, Model=OPT-13B2026.01 | 58.8 |