Question Answering on SQuAD 2.0
89.4F1RoBERTa
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| RoBERTaModel Size=Large2019.10 | 89.4 | 86.5 | — | — | — | — | |
| ElectraVerifier=false2023.07 | 89.37 | 86.47 | — | 0.97 | — | — | |
| BARTModel Size=Large2019.10 | 89.2 | 86.1 | — | — | — | — | |
| AdamWPrecision=8-bit, Backbone=RoBERTa-Large, Embedding Layer Quantization=False2023.09 | 89 | 86.1 | — | — | — | — | |
| FactorPrecision=4-bit, Backbone=RoBERTa-Large2023.09 | 88.9 | 85.9 | — | — | — | — | |
| XLNetModel Size=Large2019.10 | 88.8 | 86.1 | — | — | — | — | |
| AdamWPrecision=32-bit, Backbone=RoBERTa-Large2023.09 | 88.8 | 85.8 | — | — | — | — | |
| AdafactorPrecision=32-bit, Backbone=RoBERTa-Large, Embedding Layer Quantization=False2023.09 | 88.8 | 85.9 | — | — | — | — | |
| AdafactorPrecision=32-bit, Backbone=RoBERTa-Large2023.09 | 88.7 | 85.8 | — | — | — | — | |
| AdamWPrecision=4-bit, Backbone=RoBERTa-Large2023.09 | 88.4 | 85.4 | — | — | — | — | |
| ShadowPEFT w/ pretrained shadowTrainable Params=454,991,872, Backbone Model=Qwen3 8B + 0.5B Shadow, Inference Mode=Shadow-attached, Shadow Model Initialization=pretrained2026.04 | 87.78 | — | — | — | — | — | |
| ShadowPEFTTrainable Params=29,118,720, Backbone Model=Qwen3 8B, Inference Mode=Shadow-attached2026.04 | 87.51 | — | — | — | — | — | |
| ShadowPEFT w/ random shadowTrainable Params=454,991,872, Backbone Model=Qwen3 8B + 0.5B Shadow, Inference Mode=Shadow-attached, Shadow Model Initialization=random2026.04 | 87.39 | — | — | — | — | — | |
| LoRATrainable Params=30,670,848, Backbone Model=Qwen3 8B2026.04 | 86.9 | — | — | — | — | — | |
| ShadowPEFTTrainable Params=23,442,176, Backbone Model=Qwen3 4B, Inference Mode=Shadow-attached2026.04 | 86.84 | — | — | — | — | — | |
| DoRATrainable Params=31,039,488, Backbone Model=Qwen3 8B2026.04 | 86.79 | — | — | — | — | — | |
| FLAN T5Verifier=true2023.07 | 86.55 | 83.72 | — | 0.95 | — | — | |
| LoRATrainable Params=23,592,960, Backbone Model=Qwen3 4B2026.04 | 86.55 | — | — | — | — | — | |
| DoRATrainable Params=23,906,304, Backbone Model=Qwen3 4B2026.04 | 86.48 | — | — | — | — | — | |
| AdaLoRABackbone=DeBERTaV3-base, Trainable Parameters=0.810M2024.10 | 85.82 | 82.63 | — | — | — | — | |
| LoRABackbone=DeBERTaV3-base, Trainable Parameters=0.740M2024.10 | 85.75 | 82.56 | — | — | — | — | |
| RoCoFT-RowBackbone=DeBERTaV3-base, Trainable Parameters=0.161M2024.10 | 85.14 | 82.76 | — | — | — | — | |
| RoCoFT-ColumnBackbone=DeBERTaV3-base, Trainable Parameters=0.161M2024.10 | 85.05 | 82.6 | — | — | — | — | |
| LoHaBackbone=DeBERTaV3-base, Trainable Parameters=0.765M2024.10 | 85.01 | 81.67 | — | — | — | — | |
| FTBackbone=DeBERTaV3-base, Trainable Parameters=184M2024.10 | 83.75 | 82.92 | — | — | — | — | |
| BitfitBackbone=DeBERTaV3-base, Trainable Parameters=0.172M2024.10 | 83.75 | 79.06 | — | — | — | — | |
| FlowSteerFramework=Ours, Backbone=4o-mini2026.02 | 83.67 | 78.12 | — | — | — | — | |
| XDocPre-training Steps=100K, Pre-train (P)=true, Pre-train (D)=true2022.10 | 83.5 | — | — | — | — | — | |
| XDocPre-training Steps=100K, Pre-train (D)=true, Pre-train (W)=true2022.10 | 83.5 | — | — | — | — | — | |
| XDocPre-training Steps=1M, Pre-train (P)=true, Pre-train (D)=true, Pre-train (W)=true2022.10 | 83.5 | — | — | — | — | — | |
| UniLMModel Size=Large2019.10 | 83.4 | 80.5 | — | — | — | — | |
| RoBERTa_BASEPre-train (P)=true2022.10 | 83.4 | — | — | — | — | — | |
| XDocPre-training Steps=100K, Pre-train (P)=true2022.10 | 83.4 | — | — | — | — | — | |
| XDocPre-training Steps=100K, Pre-train (W)=true2022.10 | 83.1 | — | — | — | — | — | |
| XDocPre-training Steps=500K, Pre-train (P)=true, Pre-train (D)=true, Pre-train (W)=true2022.10 | 83.1 | — | — | — | — | — | |
| XDocPre-training Steps=100K, Pre-train (P)=true, Pre-train (W)=true2022.10 | 83 | — | — | — | — | — | |
| XDocPre-training Steps=100K, Pre-train (P)=true, Pre-train (D)=true, Pre-train (W)=true2022.10 | 83 | — | — | — | — | — | |
| XDocPre-training Steps=100K, Pre-train (D)=true2022.10 | 82.9 | — | — | — | — | — | |
| ALBERT-BASE+AA-CTPre-trained Model=ALBERT-BASE, Alignment Attention Method=CT2021.10 | 82.71 | 79.33 | — | — | — | — | |
| ALBERT-BASE+AA-GANPre-trained Model=ALBERT-BASE, Alignment Attention Method=GAN2021.10 | 82.57 | 79.25 | — | — | — | — | |
| ALBERT-BASE+AA-OTPre-trained Model=ALBERT-BASE, Alignment Attention Method=Optimal Transport (OT)2021.10 | 82.48 | 79.18 | — | — | — | — | |
| AFlowFramework=AFlow, Backbone=4o-mini2026.02 | 82.41 | 73.44 | — | — | — | — | |
| ALBERT-BASEBackbone=ALBERT-base2021.10 | 82.07 | 78.8 | — | — | — | — | |
| ALBERT-BASEPre-trained Model=ALBERT-BASE, Alignment Attention Method=None (Soft Attention)2021.10 | 82.07 | 78.8 | — | — | — | — | |
| LoKrBackbone=DeBERTaV3-base, Trainable Parameters=0.815M2024.10 | 81.96 | 80.14 | — | — | — | — | |
| BOMFBackbone=T5-base, Tuning Protocol=Freeze2024.11 | 81.82 | 76.21 | — | — | 79.01 | — | |
| BERTModel Size=Large2019.10 | 81.8 | 79 | — | — | — | — | |
| GREEDY SWABackbone=T5-base, Tuning Protocol=Freeze2024.11 | 81.73 | 76.2 | — | — | 78.97 | — | |
| LEARNED SWABackbone=T5-base, Tuning Protocol=Freeze2024.11 | 81.24 | 75.65 | — | — | 78.45 | — | |
| SWABackbone=T5-base, Tuning Protocol=Freeze2024.11 | 81.21 | 75.63 | — | — | 78.42 | — | |
| TWABackbone=T5-base, Tuning Protocol=Freeze2024.11 | 81.21 | 75.61 | — | — | 78.41 | — | |
| SM3Precision=32-bit, Backbone=RoBERTa-Large2023.09 | 81.1 | 77.2 | — | — | — | — | |
| DoRATrainable Params=9,318,400, Backbone Model=Qwen3 0.6B2026.04 | 80.91 | — | — | — | — | — | |
| BOMFBackbone=T5-base, Tuning Protocol=Full2024.11 | 80.82 | 75.79 | — | — | 78.31 | — | |
| LoRATrainable Params=9,175,040, Backbone Model=Qwen3 0.6B2026.04 | 80.75 | — | — | — | — | — | |
| LEARNED SWABackbone=T5-base, Tuning Protocol=Full2024.11 | 80.65 | 74.23 | — | — | 77.44 | — | |
| GREEDY SWABackbone=T5-base, Tuning Protocol=Full2024.11 | 80.63 | 75.44 | — | — | 78.03 | — | |
| ShadowPEFTTrainable Params=9,073,920, Backbone Model=Qwen3 0.6B, Inference Mode=Shadow-attached2026.04 | 80.54 | — | — | — | — | — | |
| SWABackbone=T5-base, Tuning Protocol=Full2024.11 | 80.31 | 74.85 | — | — | 77.58 | — | |
| TWABackbone=T5-base, Tuning Protocol=Full2024.11 | 80.29 | 74.79 | — | — | 77.54 | — | |
| Prefix-tuningBackbone=DeBERTaV3-base, Trainable Parameters=1.733M2024.10 | 79.04 | 74.94 | — | — | — | — | |
| Flan T5Verifier=false2023.07 | 79.01 | 75.72 | — | 0.83 | — | — | |
| Detached Shadow Only (pretrained)Trainable Params=454,991,872, Backbone Model=Qwen3 8B + 0.5B Shadow, Inference Mode=Detached, Shadow Model Initialization=pretrained2026.04 | 78.93 | — | — | — | — | — | |
| HPBOBackbone=T5-base, Tuning Protocol=Full2024.11 | 78.28 | 73.29 | — | — | 75.79 | — | |
| HPBOBackbone=T5-base, Tuning Protocol=Freeze2024.11 | 78.19 | 73.43 | — | — | 75.81 | — | |
| CAMEmode=fine-tuning, batch size=8k2023.07 | 77.9 | — | — | — | — | — | |
| GPT-3.5Verifier=true2023.07 | 77.63 | 67.19 | — | 0.93 | — | — | |
| CAMEmode=fine-tuning, batch size=32k2023.07 | 77.4 | — | — | — | — | — | |
| SFTFramework=SFT, Backbone=Qwen3-8B2026.02 | 77.31 | 73.44 | — | — | — | — | |
| BERT_baseBackbone=BERT-base, Padding=No2022.05 | 77.1 | — | 1 | — | — | — | |
| Prompt tuningBackbone=DeBERTaV3-base, Trainable Parameters=0.650M2024.10 | 76.72 | 73.59 | — | — | — | — | |
| Baselinemode=fine-tuning2023.07 | 76.3 | — | — | — | — | — | |
| TR-BERTBackbone=BERT-base, Padding=No2022.05 | 75.7 | — | 2.08 | — | — | — | |
| TranskimmerBackbone=BERT-base, Padding=No2022.05 | 75.7 | — | 2.1 | — | — | — | |
| OrchestratorFramework=Agent+RL, Backbone=4o-mini2026.02 | 75.24 | 70.34 | — | — | — | — | |
| AgentflowFramework=Agent+RL, Backbone=4o-mini2026.02 | 72.45 | 64.06 | — | — | — | — | |
| GRPOFramework=GRPO, Backbone=Qwen3-8B2026.02 | 72 | 66.41 | — | — | — | — | |
| DeFormerBackbone=BERT-base, Padding=Sequence2022.05 | 71.4 | — | 2.19 | — | — | — | |
| Qwen3 4BTrainable Params=0, Backbone Model=Qwen3 4B2026.04 | 68.48 | — | — | — | — | — | |
| Qwen3 8BTrainable Params=0, Backbone Model=Qwen3 8B2026.04 | 65.98 | — | — | — | — | — | |
| Router-R1Framework=Agent+RL, Backbone=4o-mini2026.02 | 65.29 | 59.84 | — | — | — | — | |
| AS-LoRADP Budget (ϵ)=32026.05 | 64.23 | 60.59 | — | — | — | — | |
| GPT-3.5Verifier=false2023.07 | 63.96 | 52.53 | — | 0.76 | — | — | |
| AS-LoRADP Budget (ϵ)=12026.05 | 61.64 | 58.09 | — | — | — | — | |
| Qwen3-8BFramework=Baseline, Backbone=Qwen3-8B2026.02 | 61.54 | 54.69 | — | — | — | — | |
| FFA-LoRADP Budget (ϵ)=32026.05 | 59.92 | 55.9 | — | — | — | — | |
| 4o-miniFramework=Baseline, Backbone=4o-mini2026.02 | 59.42 | 47.66 | — | — | — | — | |
| FFA-LoRADP Budget (ϵ)=12026.05 | 58.58 | 54.33 | — | — | — | — | |
| RoLoRADP Budget (ϵ)=32026.05 | 56.88 | 53.58 | — | — | — | — | |
| RoLoRADP Budget (ϵ)=12026.05 | 53.59 | 51.48 | — | — | — | — | |
| Qwen3 0.6BTrainable Params=0, Backbone Model=Qwen3 0.6B2026.04 | 49.47 | — | — | — | — | — | |
| Detached Shadow Only (random)Trainable Params=454,991,872, Backbone Model=Qwen3 8B + 0.5B Shadow, Inference Mode=Detached, Shadow Model Initialization=random2026.04 | 46.82 | — | — | — | — | — | |
| Detached Shadow OnlyTrainable Params=29,118,720, Backbone Model=Qwen3 8B, Inference Mode=Detached2026.04 | 44.35 | — | — | — | — | — | |
| Detached Shadow OnlyTrainable Params=9,073,920, Backbone Model=Qwen3 0.6B, Inference Mode=Detached2026.04 | 42.1 | — | — | — | — | — | |
| Detached Shadow OnlyTrainable Params=23,442,176, Backbone Model=Qwen3 4B, Inference Mode=Detached2026.04 | 41.99 | — | — | — | — | — | |
| GA-S2Backbone=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 33.352 | — | — | — | — | — | |
| AVG-S2Backbone=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 32.1499 | — | — | — | — | — | |
| GA-S2Backbone=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 31.4913 | — | — | — | — | — | |
| AVG-S2Backbone=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 31.1063 | — | — | — | — | — | |
| SMARTBackbone=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 31.001 | — | — | — | — | — |