Question Answering on SQuAD (Accuracy)
91.27AccuracySPA
Evaluation results
| Method | Links | |
|---|---|---|
| SPAModel=Qwen2.5-7B, Generator=Qwen2.5-7B (self-generated), Number of Tokens=120M2026.03 | 91.27 | |
| Active ReadingModel=Qwen2.5-7B, Generator=Qwen2.5-7B (self-generated), Number of Tokens=120M2026.03 | 90.25 | |
| QAModel=Qwen2.5-7B, Generator=Qwen2.5-7B (self-generated), Number of Tokens=120M2026.03 | 89.63 | |
| FOBackbone=Qwen3-0.6b, Fine-tuning Protocol=First-order fine-tuning2026.01 | 87.1 | |
| RephraseModel=Qwen2.5-7B, Generator=Qwen2.5-7B (self-generated), Number of Tokens=120M2026.03 | 86.86 | |
| AGZOBackbone=Qwen3-0.6b, Fine-tuning Protocol=Zeroth-order fine-tuning2026.01 | 79 | |
| LOZOBackbone=Qwen3-0.6b, Fine-tuning Protocol=Zeroth-order fine-tuning2026.01 | 78.5 | |
| MEZOBackbone=Qwen3-0.6b, Fine-tuning Protocol=Zeroth-order fine-tuning2026.01 | 77.9 | |
| SEALModel=Qwen2.5-7B, Generator=Qwen2.5-7B (self-generated), Number of Tokens=120M2026.03 | 74.23 | |
| ZeroBackbone=Qwen3-0.6b, Fine-tuning Protocol=Zero-shot prompting2026.01 | 41.6 | |
| ICLBackbone=Qwen3-0.6b, Fine-tuning Protocol=In-context learning2026.01 | 41.4 | |
| BaseModel=Qwen2.5-7B, Generator=Qwen2.5-7B (self-generated), Number of Tokens=120M2026.03 | 31.31 | |
| TransformerTraining=compute-matched, Training dataset=FineWeb-edu2026.05 | 22.96 | |
| MambaTraining=compute-matched, Training dataset=FineWeb-edu2026.05 | 10.22 | |
| SRMTraining=compute-matched, Training dataset=FineWeb-edu2026.05 | 1.47 |