Natural Language Understanding on SuperGLUE (RTE, BoolQ, WiC, CB subset)
60.8RTE AccuracyMeZO-Adam
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MeZO-AdamBackbone=OPT, Model Size=13B, Precision=BF16, Evaluation Protocol=Full-parameter fine-tuning without prompts2025.11 | 60.8 | 63.4 | 55.7 | 70.2 | 62.5 | |
| LORENBackbone=OPT, Model Size=13B, Precision=BF16, Evaluation Protocol=Full-parameter fine-tuning without prompts2025.11 | 60 | 63.2 | 57.2 | 73.7 | 63.5 | |
| HiZOOBackbone=OPT, Model Size=13B, Precision=BF16, Evaluation Protocol=Full-parameter fine-tuning without prompts2025.11 | 59.8 | 63.4 | 57.4 | 72.6 | 63.3 | |
| MeZOBackbone=LLaMA-3, Model Size=8B, Precision=BF16, Evaluation Protocol=Full-parameter fine-tuning without prompts2025.11 | 59.6 | 64.2 | 59.2 | 71.4 | 63.6 | |
| MeZOBackbone=OPT, Model Size=13B, Precision=BF16, Evaluation Protocol=Full-parameter fine-tuning without prompts2025.11 | 59.2 | 64.3 | 57.2 | 73.1 | 63.4 | |
| MeZO-SVRGBackbone=OPT, Model Size=13B, Precision=BF16, Evaluation Protocol=Full-parameter fine-tuning without prompts2025.11 | 59.1 | 65.2 | 57.3 | 69.3 | 62.7 | |
| MeZO-AdamBackbone=LLaMA-3, Model Size=8B, Precision=BF16, Evaluation Protocol=Full-parameter fine-tuning without prompts2025.11 | 58.6 | 63.7 | 57.7 | 68.5 | 62.1 | |
| LORENBackbone=LLaMA-3, Model Size=8B, Precision=BF16, Evaluation Protocol=Full-parameter fine-tuning without prompts2025.11 | 58.6 | 64.8 | 59.2 | 72.2 | 63.7 | |
| LOZOBackbone=OPT, Model Size=13B, Precision=BF16, Evaluation Protocol=Full-parameter fine-tuning without prompts2025.11 | 58.5 | 63.2 | 57.6 | 68.5 | 62 | |
| MeZO-SVRGBackbone=LLaMA-3, Model Size=8B, Precision=BF16, Evaluation Protocol=Full-parameter fine-tuning without prompts2025.11 | 58.2 | 63.8 | 58 | 65.7 | 61.4 | |
| HiZOOBackbone=LLaMA-3, Model Size=8B, Precision=BF16, Evaluation Protocol=Full-parameter fine-tuning without prompts2025.11 | 57.8 | 63.7 | 59.2 | 66.7 | 61.8 | |
| LOZOBackbone=LLaMA-3, Model Size=8B, Precision=BF16, Evaluation Protocol=Full-parameter fine-tuning without prompts2025.11 | 57 | 64.8 | 57.2 | 71.4 | 62.6 |