Science Question Answering on OpenBookQA
94.6AccuracyEVOSELECT
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| EVOSELECTIteration=Iter. 2, Selection Ratio=0.2, Model Scale=14b base model2026.04 | 94.6 | — | — | — | — | — | — | — | |
| AttributionIteration=Iter. 1, Selection Ratio=0.2, Model Scale=14b base model2026.04 | 94.4 | — | — | — | — | — | — | — | |
| RandomIteration=Iter. 2, Selection Ratio=0.2, Model Scale=14b base model2026.04 | 94.4 | — | — | — | — | — | — | — | |
| TSDSIteration=Iter. 2, Selection Ratio=0.5, Model Scale=14b base model2026.04 | 94.4 | — | — | — | — | — | — | — | |
| EVOSELECTIteration=Iter. 2, Selection Ratio=0.5, Model Scale=14b base model2026.04 | 94.4 | — | — | — | — | — | — | — | |
| EVOSELECTIteration=Iter. 1, Selection Ratio=0.5, Model Scale=14b base model2026.04 | 94.2 | — | — | — | — | — | — | — | |
| RandomIteration=Iter. 1, Selection Ratio=0.5, Model Scale=14b base model2026.04 | 93.4 | — | — | — | — | — | — | — | |
| AttributionIteration=Iter. 2, Selection Ratio=0.5, Model Scale=14b base model2026.04 | 93.4 | — | — | — | — | — | — | — | |
| DiversityIteration=Iter. 2, Selection Ratio=0.5, Model Scale=14b base model2026.04 | 93.4 | — | — | — | — | — | — | — | |
| AttributionIteration=Iter. 1, Selection Ratio=0.5, Model Scale=14b base model2026.04 | 93.2 | — | — | — | — | — | — | — | |
| DiversityIteration=Iter. 1, Selection Ratio=0.5, Model Scale=14b base model2026.04 | 93.2 | — | — | — | — | — | — | — | |
| RandomIteration=Iter. 2, Selection Ratio=0.5, Model Scale=14b base model2026.04 | 93.2 | — | — | — | — | — | — | — | |
| RandomIteration=Iter. 1, Selection Ratio=0.2, Model Scale=14b base model2026.04 | 93 | — | — | — | — | — | — | — | |
| EVOSELECTIteration=Iter. 1, Selection Ratio=0.2, Model Scale=14b base model2026.04 | 93 | — | — | — | — | — | — | — | |
| AttributionIteration=Iter. 2, Selection Ratio=0.2, Model Scale=14b base model2026.04 | 93 | — | — | — | — | — | — | — | |
| AllIteration=Iter. 1, Model Scale=14b base model2026.04 | 92.6 | — | — | — | — | — | — | — | |
| Attr-DivIteration=Iter. 1, Selection Ratio=0.2, Model Scale=14b base model2026.04 | 92.6 | — | — | — | — | — | — | — | |
| TSDSIteration=Iter. 1, Selection Ratio=0.5, Model Scale=14b base model2026.04 | 92.4 | — | — | — | — | — | — | — | |
| Attr-DivIteration=Iter. 2, Selection Ratio=0.5, Model Scale=14b base model2026.04 | 92.4 | — | — | — | — | — | — | — | |
| TSDSIteration=Iter. 2, Selection Ratio=0.2, Model Scale=14b base model2026.04 | 92.4 | — | — | — | — | — | — | — | |
| DiversityIteration=Iter. 1, Selection Ratio=0.2, Model Scale=14b base model2026.04 | 92.2 | — | — | — | — | — | — | — | |
| Attr-DivIteration=Iter. 1, Selection Ratio=0.5, Model Scale=14b base model2026.04 | 92.2 | — | — | — | — | — | — | — | |
| TSDSIteration=Iter. 1, Selection Ratio=0.2, Model Scale=14b base model2026.04 | 92.2 | — | — | — | — | — | — | — | |
| Attr-DivIteration=Iter. 2, Selection Ratio=0.2, Model Scale=14b base model2026.04 | 92.2 | — | — | — | — | — | — | — | |
| DiversityIteration=Iter. 2, Selection Ratio=0.2, Model Scale=14b base model2026.04 | 91.8 | — | — | — | — | — | — | — | |
| AllIteration=Iter. 2, Model Scale=14b base model2026.04 | 91.6 | — | — | — | — | — | — | — | |
| BaseModel Scale=14b base model2026.04 | 91.4 | — | — | — | — | — | — | — | |
| STaR-GATE-DMode=Thinking Mode2025.06 | 69.03 | 20.8 | 5.21 | 6.64 | 5.4 | 4.73 | 4.35 | 5.7 | |
| SwiftMode=Thinking Mode2025.06 | 68.14 | 96.5 | 6.16 | 6.86 | 6.49 | 5.89 | 5.99 | 6.52 | |
| STaR-GATEMode=Thinking Mode2025.06 | 65.49 | 92.5 | 6.01 | 6.68 | 6.35 | 5.8 | 5.78 | 6.36 | |
| RefitMode=Thinking Mode2025.06 | 65.04 | 96.5 | 5.84 | 6.63 | 6.12 | 5.58 | 5.62 | 6.25 | |
| DPOMode=Thinking Mode2025.06 | 61.95 | 10.6 | 5.09 | 6.21 | 5.35 | 4.82 | 4.47 | 5.55 | |
| StepDPOMode=Thinking Mode2025.06 | 61.06 | 11.5 | 4.9 | 6.14 | 5.06 | 4.56 | 4.29 | 5.33 | |
| DEEPSEEK-V2-LITEPARAMS=16B, Variant=ORIGINAL2025.11 | 45.2 | — | — | — | — | — | — | — | |
| LLAMA-4-SCOUTPARAMS=109B, Variant=ORIGINAL2025.11 | 45.2 | — | — | — | — | — | — | — | |
| QWEN-3-30B MOEPARAMS=30B, Variant=ORIGINAL2025.11 | 45 | — | — | — | — | — | — | — | |
| LLAMA-4-SCOUTPARAMS=109B, Variant=FCSD2025.11 | 44.4 | — | — | — | — | — | — | — | |
| LLAMA-4-SCOUTPARAMS=109B, Variant=SFT2025.11 | 43.8 | — | — | — | — | — | — | — | |
| MobileMoE-LActive Parameters=922M, Total Parameters=5.3B, Few-shot count=0-shot2026.05 | 42.8 | — | — | — | — | — | — | — | |
| QWEN-3-30B MOEPARAMS=30B, Variant=FCSD2025.11 | 42.8 | — | — | — | — | — | — | — | |
| DEEPSEEK-V2-LITEPARAMS=16B, Variant=FCSD2025.11 | 41.8 | — | — | — | — | — | — | — | |
| QWEN-3-30B MOEPARAMS=30B, Variant=SFT2025.11 | 41.4 | — | — | — | — | — | — | — | |
| DEEPSEEK-V2-LITEPARAMS=16B, Variant=SFT2025.11 | 41.2 | — | — | — | — | — | — | — | |
| CLIMBSize=950M2025.04 | 41.2 | — | — | — | — | — | — | — | |
| MobileMoE-MActive Parameters=528M, Total Parameters=2.8B, Few-shot count=0-shot2026.05 | 39.4 | — | — | — | — | — | — | — | |
| SmolLMSize=360M2025.04 | 37.6 | — | — | — | — | — | — | — | |
| Llama-3.2Size=1.2B2025.04 | 37.2 | — | — | — | — | — | — | — | |
| CLIMBSize=350M2025.04 | 36.6 | — | — | — | — | — | — | — | |
| BaseMode=Thinking Mode2025.06 | 36.28 | 74.3 | 5.99 | 6.77 | 6.15 | 5.43 | 5.95 | 6.31 | |
| TinyLlamaSize=1.1B2025.04 | 36 | — | — | — | — | — | — | — | |
| AMD-OLMoSize=1.2B2025.04 | 35.8 | — | — | — | — | — | — | — | |
| Qwen2.5Size=490M2025.04 | 35.2 | — | — | — | — | — | — | — | |
| MobileMoE-SActive Parameters=272M, Total Parameters=1.3B, Few-shot count=0-shot2026.05 | 34.6 | — | — | — | — | — | — | — | |
| X-GRAM-50%Backbone Scale=1.15B Scale, Cfg=4x, # Params=1.15B+7.47B, Evaluation Protocol=Zero/Few-shot2026.04 | 33.4 | — | — | — | — | — | — | — | |
| X-GRAM-50%Backbone Scale=0.73B Scale, Cfg=4x, # Params=0.73B+4.67B, Evaluation Protocol=Zero/Few-shot2026.04 | 32 | — | — | — | — | — | — | — | |
| X-GRAM-100%Backbone Scale=1.15B Scale, Cfg=2x, # Params=1.15B+7.47B, Evaluation Protocol=Zero/Few-shot2026.04 | 31.6 | — | — | — | — | — | — | — | |
| X-GRAM-50%Backbone Scale=1.15B Scale, Cfg=2x, # Params=1.15B+3.73B, Evaluation Protocol=Zero/Few-shot2026.04 | 31 | — | — | — | — | — | — | — | |
| X-GRAM-100%Backbone Scale=1.15B Scale, Cfg=1x, # Params=1.15B+3.73B, Evaluation Protocol=Zero/Few-shot2026.04 | 30.8 | — | — | — | — | — | — | — | |
| X-GRAM-100%Backbone Scale=1.15B Scale, Cfg=4x, # Params=1.15B+14.94B, Evaluation Protocol=Zero/Few-shot2026.04 | 30.8 | — | — | — | — | — | — | — | |
| MDMZero-shot=true2026.05 | 30.8 | — | — | — | — | — | — | — | |
| X-GRAM-100%Backbone Scale=0.73B Scale, Cfg=2x, # Params=0.73B+4.67B, Evaluation Protocol=Zero/Few-shot2026.04 | 30.4 | — | — | — | — | — | — | — | |
| MoRTBackbone Scale=1.15B Scale, Cfg=2x, # Params=1.15B+7.47B, Evaluation Protocol=Zero/Few-shot2026.04 | 30.4 | — | — | — | — | — | — | — | |
| X-GRAM-100%Backbone Scale=0.73B Scale, Cfg=4x, # Params=0.73B+9.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 30.2 | — | — | — | — | — | — | — | |
| EngramBackbone Scale=1.15B Scale, Cfg=4x, # Params=1.15B+14.94B, Evaluation Protocol=Zero/Few-shot2026.04 | 29.8 | — | — | — | — | — | — | — | |
| EngramBackbone Scale=0.73B Scale, Cfg=2x, # Params=0.73B+4.67B, Evaluation Protocol=Zero/Few-shot2026.04 | 29.6 | — | — | — | — | — | — | — | |
| MoRTBackbone Scale=0.73B Scale, Cfg=4x, # Params=0.73B+9.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 29.6 | — | — | — | — | — | — | — | |
| BaselineBackbone Scale=1.15B Scale, Cfg=N/A, # Params=1.15B, Evaluation Protocol=Zero/Few-shot2026.04 | 29.2 | — | — | — | — | — | — | — | |
| RetokenBackbone Scale=0.73B Scale, Cfg=2x, # Params=0.73B+4.67B, Evaluation Protocol=Zero/Few-shot2026.04 | 29 | — | — | — | — | — | — | — | |
| EngramBackbone Scale=0.73B Scale, Cfg=4x, # Params=0.73B+9.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 29 | — | — | — | — | — | — | — | |
| X-GRAM-50%Backbone Scale=0.73B Scale, Cfg=1x, # Params=0.73B+1.17B, Evaluation Protocol=Zero/Few-shot2026.04 | 28.8 | — | — | — | — | — | — | — | |
| MoRTBackbone Scale=0.73B Scale, Cfg=2x, # Params=0.73B+4.67B, Evaluation Protocol=Zero/Few-shot2026.04 | 28.8 | — | — | — | — | — | — | — | |
| RetokenBackbone Scale=1.15B Scale, Cfg=1x, # Params=1.15B+3.73B, Evaluation Protocol=Zero/Few-shot2026.04 | 28.6 | — | — | — | — | — | — | — | |
| MoRTBackbone Scale=1.15B Scale, Cfg=1x, # Params=1.15B+3.73B, Evaluation Protocol=Zero/Few-shot2026.04 | 28.6 | — | — | — | — | — | — | — | |
| X-GRAM-50%Backbone Scale=1.15B Scale, Cfg=1x, # Params=1.15B+1.87B, Evaluation Protocol=Zero/Few-shot2026.04 | 28.6 | — | — | — | — | — | — | — | |
| RetokenBackbone Scale=1.15B Scale, Cfg=2x, # Params=1.15B+7.47B, Evaluation Protocol=Zero/Few-shot2026.04 | 28.6 | — | — | — | — | — | — | — | |
| RetokenBackbone Scale=1.15B Scale, Cfg=4x, # Params=1.15B+14.94B, Evaluation Protocol=Zero/Few-shot2026.04 | 28.4 | — | — | — | — | — | — | — | |
| LoopMDMS=1, Zero-shot=true2026.05 | 28.2 | — | — | — | — | — | — | — | |
| RetokenBackbone Scale=0.73B Scale, Cfg=1x, # Params=0.73B+2.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 28 | — | — | — | — | — | — | — | |
| MoRTBackbone Scale=0.73B Scale, Cfg=1x, # Params=0.73B+2.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 28 | — | — | — | — | — | — | — | |
| EngramBackbone Scale=1.15B Scale, Cfg=1x, # Params=1.15B+3.73B, Evaluation Protocol=Zero/Few-shot2026.04 | 28 | — | — | — | — | — | — | — | |
| LoopMDMS=6, Zero-shot=true2026.05 | 28 | — | — | — | — | — | — | — | |
| LoopMDMS=12, Zero-shot=true2026.05 | 28 | — | — | — | — | — | — | — | |
| EngramBackbone Scale=0.73B Scale, Cfg=1x, # Params=0.73B+2.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 27.8 | — | — | — | — | — | — | — | |
| BaselineBackbone Scale=0.73B Scale, Cfg=N/A, # Params=0.73B, Evaluation Protocol=Zero/Few-shot2026.04 | 27.6 | — | — | — | — | — | — | — | |
| X-GRAM-50%Backbone Scale=0.73B Scale, Cfg=2x, # Params=0.73B+2.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 27.4 | — | — | — | — | — | — | — | |
| EngramBackbone Scale=1.15B Scale, Cfg=2x, # Params=1.15B+7.47B, Evaluation Protocol=Zero/Few-shot2026.04 | 27.4 | — | — | — | — | — | — | — | |
| MoRTBackbone Scale=1.15B Scale, Cfg=4x, # Params=1.15B+14.94B, Evaluation Protocol=Zero/Few-shot2026.04 | 27.2 | — | — | — | — | — | — | — | |
| X-GRAM-100%Backbone Scale=0.73B Scale, Cfg=1x, # Params=0.73B+2.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 26.8 | — | — | — | — | — | — | — | |
| RetokenBackbone Scale=0.73B Scale, Cfg=4x, # Params=0.73B+9.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 25.8 | — | — | — | — | — | — | — |