Natural Language Inference on RTE (Accuracy)
79Accuracy (RTE)GRZO
Evaluation results
| Method | Links | |
|---|---|---|
| GRZOBackbone=RoBERTa-large (350M), k-shot setting=512, Optimization Protocol=Zeroth-Order, Training Steps=20k, Batch Size=16, Precision=FP162026.06 | 79 | |
| MeZOBackbone=RoBERTa-large (350M), k-shot setting=512, Optimization Protocol=Zeroth-Order, Training Steps=20k, Batch Size=16, Precision=FP162026.06 | 78.6 | |
| FZOOBackbone=RoBERTa-large (350M), k-shot setting=512, Optimization Protocol=Zeroth-Order, Training Steps=20k, Batch Size=16, Precision=FP162026.06 | 78.1 | |
| LP(FO)Backbone=RoBERTa-large (350M), k-shot setting=512, Optimization Protocol=Linear Probing (First-Order), Training Steps=20k, Batch Size=16, Precision=FP162026.06 | 73.1 | |
| FT(FO)Backbone=RoBERTa-large (350M), k-shot setting=512, Optimization Protocol=Fine-Tuning (First-Order), Training Steps=20k, Batch Size=16, Precision=FP162026.06 | 66.4 | |
| Hybrid-LoRAModel=OPT-1.3b, Optimizer=SGD2026.04 | 61 | |
| FO-LoRAModel=OPT-1.3b, Optimizer=Adam2026.04 | 58.6 | |
| FO-LoRAModel=OPT-1.3b, Optimizer=SGD2026.04 | 54.8 | |
| Zero-shotBackbone=RoBERTa-large (350M), k-shot setting=5122026.06 | 51.4 |