List your company
ResearchFormerly WizWand

Question Answering on SQuAD (F1)

94.7F1 Score32-bit Adafactor+
21.827240.746159.66578.5839Feb 16, 2021Dec 30, 2021Nov 12, 2022Sep 26, 2023Aug 8, 2024Jun 21, 2025May 5, 2026
Updated 6mo ago

Evaluation results

MethodLinks
32-bit Adafactor+Backbone=RoBERTa-L, Precision=32-bit, beta1=02023.0994.7
32-bit AdamWBackbone=RoBERTa-L, Precision=32-bit2023.0994.6
32-bit AdafactorBackbone=RoBERTa-L, Precision=32-bit2023.0994.6
4-bit FactorBackbone=RoBERTa-L, Precision=4-bit2023.0994.6
8-bit AdamW†Backbone=RoBERTa-L, Precision=8-bit, Quantization Policy=Do not quantize optimizer states for embedding layers2023.0994.5
4-bit AdamWBackbone=RoBERTa-L, Precision=4-bit2023.0994.5
32-bit SM3Backbone=RoBERTa-L, Precision=32-bit2023.0991.7
BaselineModel=Bert-Large2021.0291.1
2:4 Transposable MaskModel=Bert-Large2021.0290.18
ConMeZOModel=OPT-13B2025.1183.66
ZO-AdamBackbone=Qwen-3, Model Size=8B, Fine-tuning=16-rank LoRA2026.0583.16
MEAZOBackbone=Qwen-3, Model Size=8B, Fine-tuning=16-rank LoRA2026.0582.94
RAdaZOBackbone=Qwen-3, Model Size=8B, Fine-tuning=16-rank LoRA2026.0582.88
ZO-SGDBackbone=Qwen-3, Model Size=8B, Fine-tuning=16-rank LoRA2026.0582.69
MeZOModel=OPT-13B2025.1182.28
Tri-RAGGenerator Backbone=Qwen-Long2026.0479.2
Tri-RAGGenerator Backbone=GPT-4.1-nano2026.0477.1
AflowGenerator Backbone=Qwen-Long2026.0477.1
Tri-RAGGenerator Backbone=Qwen-Max2026.0476.5
ZO-AdamBackbone=Llama-3.2, Model Size=1B, Fine-tuning=16-rank LoRA2026.0576.24
ZO-SGDBackbone=Llama-3.2, Model Size=1B, Fine-tuning=16-rank LoRA2026.0576.16
Tri-RAGGenerator Backbone=Gemma-2-27B2026.0475.8
MEAZOBackbone=Llama-3.2, Model Size=1B, Fine-tuning=16-rank LoRA2026.0575.77
ConMeZOModel=OPT-1.3B2025.1175.34
Tri-RAGGenerator Backbone=GPT-4o-mini2026.0475.3
RAdaZOBackbone=Llama-3.2, Model Size=1B, Fine-tuning=16-rank LoRA2026.0574.54
AflowGenerator Backbone=Qwen-Max2026.0474
AflowGenerator Backbone=GPT-4o-mini2026.0473.7
MeZOModel=OPT-1.3B2025.1172.76
AflowGenerator Backbone=GPT-4.1-nano2026.0472.1
AflowGenerator Backbone=Gemma-2-27B2026.0471.9
Self-ActGenerator Backbone=Qwen-Long2026.0470.4
Self-ActGenerator Backbone=GPT-4o-mini2026.0469.5
RAdaZOBackbone=Qwen-3, Model Size=0.6B, Fine-tuning=16-rank LoRA2026.0569.5
ZO-SGDBackbone=Qwen-3, Model Size=0.6B, Fine-tuning=16-rank LoRA2026.0569.3
ZO-AdamBackbone=Qwen-3, Model Size=0.6B, Fine-tuning=16-rank LoRA2026.0568.91
MEAZOBackbone=Qwen-3, Model Size=0.6B, Fine-tuning=16-rank LoRA2026.0568.72
Self-ActGenerator Backbone=Qwen-Max2026.0468.3
Self-ActGenerator Backbone=Gemma-2-27B2026.0465.7
Self-ActGenerator Backbone=GPT-4.1-nano2026.0462.7
ReActGenerator Backbone=Qwen-Long2026.0458.7
MEAZOBackbone=Qwen-3 0.6B, Tuning=20-token prompt tuning2026.0556.46
ReActGenerator Backbone=GPT-4o-mini2026.0455.3
ReActGenerator Backbone=Qwen-Max2026.0454.5
ReActGenerator Backbone=Gemma-2-27B2026.0454.5
DirectGenerator Backbone=Qwen-Long2026.0454.3
ZO-AdamBackbone=Qwen-3 0.6B, Tuning=20-token prompt tuning2026.0554.28
ZO-SGDBackbone=Qwen-3 0.6B, Tuning=20-token prompt tuning2026.0552.44
DirectGenerator Backbone=GPT-4o-mini2026.0451.2
ReActGenerator Backbone=GPT-4.1-nano2026.0450.9
ZO-SGDBackbone=Llama-3.2 1B, Tuning=20-token prompt tuning2026.0549.98
RAdaZOBackbone=Llama-3.2 1B, Tuning=20-token prompt tuning2026.0549.75
DirectGenerator Backbone=GPT-4.1-nano2026.0449.3
ZO-AdamBackbone=Llama-3.2 1B, Tuning=20-token prompt tuning2026.0548.87
DirectGenerator Backbone=Qwen-Max2026.0448.5
DirectGenerator Backbone=Gemma-2-27B2026.0448.5
MEAZOBackbone=Llama-3.2 1B, Tuning=20-token prompt tuning2026.0547.78
RAdaZOBackbone=Qwen-3 0.6B, Tuning=20-token prompt tuning2026.0543.3
BaseBackbone=Qwen-3, Model Size=0.6B, Fine-tuning=None (Before tuning)2026.0531.76
BaseBackbone=Qwen-3 0.6B, Tuning=20-token prompt tuning2026.0531.76
BaseBackbone=Qwen-3, Model Size=8B, Fine-tuning=None (Before tuning)2026.0531.5
BaseBackbone=Llama-3.2, Model Size=1B, Fine-tuning=None (Before tuning)2026.0524.63
BaseBackbone=Llama-3.2 1B, Tuning=20-token prompt tuning2026.0524.63