Question Answering on SQuAD (F1)
94.7F1 Score32-bit Adafactor+
Evaluation Results
| Method | Links | |
|---|---|---|
| 32-bit Adafactor+Backbone=RoBERTa-L, Precision=32-bit, beta1=02023.09 | 94.7 | |
| 32-bit AdamWBackbone=RoBERTa-L, Precision=32-bit2023.09 | 94.6 | |
| 32-bit AdafactorBackbone=RoBERTa-L, Precision=32-bit2023.09 | 94.6 | |
| 4-bit FactorBackbone=RoBERTa-L, Precision=4-bit2023.09 | 94.6 | |
| 8-bit AdamW†Backbone=RoBERTa-L, Precision=8-bit, Quantization Policy=Do not quantize optimizer states for embedding layers2023.09 | 94.5 | |
| 4-bit AdamWBackbone=RoBERTa-L, Precision=4-bit2023.09 | 94.5 | |
| 32-bit SM3Backbone=RoBERTa-L, Precision=32-bit2023.09 | 91.7 | |
| BaselineModel=Bert-Large2021.02 | 91.1 | |
| 2:4 Transposable MaskModel=Bert-Large2021.02 | 90.18 | |
| ConMeZOModel=OPT-13B2025.11 | 83.66 | |
| ZO-AdamBackbone=Qwen-3, Model Size=8B, Fine-tuning=16-rank LoRA2026.05 | 83.16 | |
| MEAZOBackbone=Qwen-3, Model Size=8B, Fine-tuning=16-rank LoRA2026.05 | 82.94 | |
| RAdaZOBackbone=Qwen-3, Model Size=8B, Fine-tuning=16-rank LoRA2026.05 | 82.88 | |
| ZO-SGDBackbone=Qwen-3, Model Size=8B, Fine-tuning=16-rank LoRA2026.05 | 82.69 | |
| MeZOModel=OPT-13B2025.11 | 82.28 | |
| Tri-RAGGenerator Backbone=Qwen-Long2026.04 | 79.2 | |
| Tri-RAGGenerator Backbone=GPT-4.1-nano2026.04 | 77.1 | |
| AflowGenerator Backbone=Qwen-Long2026.04 | 77.1 | |
| Tri-RAGGenerator Backbone=Qwen-Max2026.04 | 76.5 | |
| ZO-AdamBackbone=Llama-3.2, Model Size=1B, Fine-tuning=16-rank LoRA2026.05 | 76.24 | |
| ZO-SGDBackbone=Llama-3.2, Model Size=1B, Fine-tuning=16-rank LoRA2026.05 | 76.16 | |
| Tri-RAGGenerator Backbone=Gemma-2-27B2026.04 | 75.8 | |
| MEAZOBackbone=Llama-3.2, Model Size=1B, Fine-tuning=16-rank LoRA2026.05 | 75.77 | |
| ConMeZOModel=OPT-1.3B2025.11 | 75.34 | |
| Tri-RAGGenerator Backbone=GPT-4o-mini2026.04 | 75.3 | |
| RAdaZOBackbone=Llama-3.2, Model Size=1B, Fine-tuning=16-rank LoRA2026.05 | 74.54 | |
| AflowGenerator Backbone=Qwen-Max2026.04 | 74 | |
| AflowGenerator Backbone=GPT-4o-mini2026.04 | 73.7 | |
| MeZOModel=OPT-1.3B2025.11 | 72.76 | |
| AflowGenerator Backbone=GPT-4.1-nano2026.04 | 72.1 | |
| AflowGenerator Backbone=Gemma-2-27B2026.04 | 71.9 | |
| Self-ActGenerator Backbone=Qwen-Long2026.04 | 70.4 | |
| Self-ActGenerator Backbone=GPT-4o-mini2026.04 | 69.5 | |
| RAdaZOBackbone=Qwen-3, Model Size=0.6B, Fine-tuning=16-rank LoRA2026.05 | 69.5 | |
| ZO-SGDBackbone=Qwen-3, Model Size=0.6B, Fine-tuning=16-rank LoRA2026.05 | 69.3 | |
| ZO-AdamBackbone=Qwen-3, Model Size=0.6B, Fine-tuning=16-rank LoRA2026.05 | 68.91 | |
| MEAZOBackbone=Qwen-3, Model Size=0.6B, Fine-tuning=16-rank LoRA2026.05 | 68.72 | |
| Self-ActGenerator Backbone=Qwen-Max2026.04 | 68.3 | |
| Self-ActGenerator Backbone=Gemma-2-27B2026.04 | 65.7 | |
| Self-ActGenerator Backbone=GPT-4.1-nano2026.04 | 62.7 | |
| ReActGenerator Backbone=Qwen-Long2026.04 | 58.7 | |
| MEAZOBackbone=Qwen-3 0.6B, Tuning=20-token prompt tuning2026.05 | 56.46 | |
| ReActGenerator Backbone=GPT-4o-mini2026.04 | 55.3 | |
| ReActGenerator Backbone=Qwen-Max2026.04 | 54.5 | |
| ReActGenerator Backbone=Gemma-2-27B2026.04 | 54.5 | |
| DirectGenerator Backbone=Qwen-Long2026.04 | 54.3 | |
| ZO-AdamBackbone=Qwen-3 0.6B, Tuning=20-token prompt tuning2026.05 | 54.28 | |
| ZO-SGDBackbone=Qwen-3 0.6B, Tuning=20-token prompt tuning2026.05 | 52.44 | |
| DirectGenerator Backbone=GPT-4o-mini2026.04 | 51.2 | |
| ReActGenerator Backbone=GPT-4.1-nano2026.04 | 50.9 | |
| ZO-SGDBackbone=Llama-3.2 1B, Tuning=20-token prompt tuning2026.05 | 49.98 | |
| RAdaZOBackbone=Llama-3.2 1B, Tuning=20-token prompt tuning2026.05 | 49.75 | |
| DirectGenerator Backbone=GPT-4.1-nano2026.04 | 49.3 | |
| ZO-AdamBackbone=Llama-3.2 1B, Tuning=20-token prompt tuning2026.05 | 48.87 | |
| DirectGenerator Backbone=Qwen-Max2026.04 | 48.5 | |
| DirectGenerator Backbone=Gemma-2-27B2026.04 | 48.5 | |
| MEAZOBackbone=Llama-3.2 1B, Tuning=20-token prompt tuning2026.05 | 47.78 | |
| RAdaZOBackbone=Qwen-3 0.6B, Tuning=20-token prompt tuning2026.05 | 43.3 | |
| BaseBackbone=Qwen-3, Model Size=0.6B, Fine-tuning=None (Before tuning)2026.05 | 31.76 | |
| BaseBackbone=Qwen-3 0.6B, Tuning=20-token prompt tuning2026.05 | 31.76 | |
| BaseBackbone=Qwen-3, Model Size=8B, Fine-tuning=None (Before tuning)2026.05 | 31.5 | |
| BaseBackbone=Llama-3.2, Model Size=1B, Fine-tuning=None (Before tuning)2026.05 | 24.63 | |
| BaseBackbone=Llama-3.2 1B, Tuning=20-token prompt tuning2026.05 | 24.63 |