Multiple Choice on COPA
100AccuracyFull Fine-Tuning (FT)
Evaluation Results
| Method | Links | |
|---|---|---|
| Full Fine-Tuning (FT)Model=Qwen2.5-14B, Trainable Parameters=14.7B2025.12 | 100 | |
| LoRAModel=Qwen2.5-14B, Trainable Parameters=6.3M2025.12 | 100 | |
| Partial-LoRAModel=Qwen2.5-14B, Trainable Parameters=122K2025.12 | 100 | |
| Masking (0.001%)Model=Qwen2.5-14B, Trainable Parameters=149K2025.12 | 98 | |
| Partial-LoRAModel=LLAMA2-7B, Trainable Parameters=80K2025.12 | 90 | |
| Full Fine-Tuning (FT)Model=LLAMA3.1-8B, Trainable Parameters=8B2025.12 | 90 | |
| LoRAModel=LLAMA3.1-8B, Trainable Parameters=3.4M2025.12 | 89 | |
| Masking (0.001%)Model=LLAMA2-7B, Trainable Parameters=68K2025.12 | 88 | |
| Partial-LoRAModel=LLAMA3.1-8B, Trainable Parameters=92k2025.12 | 88 | |
| AR1-ZOModel=Qwen3-32B, Rank (r)=642026.05 | 88 | |
| MeZO LoRAModel Backbone=Llama2-7B, Fine-tuning Strategy=LoRA, Optimization Algorithm=MeZO, Number of training examples=10002026.06 | 88 | |
| Full Fine-Tuning (FT)Model=LLAMA2-7B, Trainable Parameters=6.7B2025.12 | 87 | |
| Dominant-layer ZO FTModel Backbone=Llama2-7B, Fine-tuning Strategy=Full Fine-tuning, Optimization Algorithm=Dominant-layer ZO, Number of training examples=10002026.06 | 87 | |
| Dominant-layer ZO LoRAModel Backbone=Llama2-7B, Fine-tuning Strategy=LoRA, Optimization Algorithm=Dominant-layer ZO, Number of training examples=10002026.06 | 87 | |
| LOZOModel=OPT-13B2026.05 | 86 | |
| AR1-ZOModel=OPT-13B, Rank (r)=642026.05 | 86 | |
| First Order Adamw FTModel Backbone=Llama2-7B, Fine-tuning Strategy=Full Fine-tuning, Optimization Algorithm=First Order Adamw, Number of training examples=10002026.06 | 86 | |
| MeZO FTModel Backbone=Llama2-7B, Fine-tuning Strategy=Full Fine-tuning, Optimization Algorithm=MeZO, Number of training examples=10002026.06 | 86 | |
| LoRAModel=LLAMA2-7B, Trainable Parameters=4.2M2025.12 | 85 | |
| MeZO-LoRAModel=OPT-13B, Rank (r)=642026.05 | 84 | |
| FT (Adam)Model=OPT-2.7B2026.05 | 81 | |
| Zero-shot w/o finetuneModel Backbone=Llama2-7B, Fine-tuning Strategy=None, Optimization Algorithm=None, Number of training examples=10002026.06 | 81 | |
| LOZOModel=OPT-2.7B2026.05 | 80 | |
| AR1-ZOModel=OPT-2.7B, Rank (r)=642026.05 | 79 | |
| FT (Adam)Model=OPT-13B2026.05 | 79 | |
| CAQ-ZOBackbone=Llama-2-7B, Precision=NF42026.05 | 78.4 | |
| MeZOBackbone=Llama-2-7B, Precision=BF162026.05 | 78 | |
| Zero-Shot-QBackbone=Llama-2-7B, Precision=NF42026.05 | 78 | |
| QuZOBackbone=Llama-2-7B, Precision=NF42026.05 | 77.9 | |
| MeZO-LoRAModel=Qwen3-32B, Rank (r)=642026.05 | 77 | |
| CAQ-ZOBackbone=Qwen-2.5-1.5B, Precision=NF42026.05 | 76.2 | |
| Masking (0.001%)Model=LLAMA3.1-8B, Trainable Parameters=80K2025.12 | 76 | |
| MeZOBackbone=Qwen-2.5-1.5B, Precision=BF162026.05 | 75.1 | |
| ZO-Alt-NaiveModel=OPT-13B2026.05 | 75 | |
| Zero-Shot-QBackbone=Qwen-2.5-1.5B, Precision=NF42026.05 | 73 | |
| ZO-Alt-NaiveModel=Qwen3-32B2026.05 | 73 | |
| MeZO-LoRAModel=OPT-2.7B, Rank (r)=642026.05 | 72 | |
| AR1-ZOModel=Qwen3-1.7B, Rank (r)=642026.05 | 72 | |
| ZO-Alt-NaiveModel=OPT-2.7B2026.05 | 71 | |
| MeZO-LoRAModel=Qwen3-1.7B, Rank (r)=642026.05 | 71 | |
| ZO-Alt-NaiveModel=Qwen3-1.7B2026.05 | 71 | |
| QuZOBackbone=Qwen-2.5-1.5B, Precision=NF42026.05 | 63.9 |