List your company
ResearchFormerly WizWand

Reading Comprehension on DROP (DROP metric)

48.68DROP ScoreMONA
14.900823.670432.4441.2096Mar 17, 2026Mar 28, 2026Apr 9, 2026Apr 21, 2026May 2, 2026May 14, 2026May 26, 2026
Updated 5mo ago

Evaluation results

MethodLinks
MONAModel=MOE-68B-A3B, Training Tokens=700B, Evaluation Mode=Chain of Thought (CoT)2026.0548.68
MuonModel=MOE-68B-A3B, Training Tokens=700B, Evaluation Mode=Chain of Thought (CoT)2026.0547.57
AdamWModel=MOE-68B-A3B, Training Tokens=700B, Evaluation Mode=Chain of Thought (CoT)2026.0545.6
Warmup-Stable-Only (WSO)Model=8B, αpre=1.0, αmid=1.02026.0336.4
WSDModel=8B, αpre=1.0, αmid=0.02026.0332.6
LinearModel=8B, αpre=0.1, αmid=1.02026.0330.5
WSDModel=8B, αpre=0.1, αmid=1.02026.0329.9
CosineModel=8B, αpre=0.1, αmid=1.02026.0328.8
WSDModel=8B, αpre=0.1, αmid=0.02026.0328.3
LinearModel=8B, αpre=0.1, αmid=0.02026.0328.3
CosineModel=8B, αpre=0.1, αmid=0.02026.0326.9
HYPERCONNECTIONSInput length=2k tokens, Evaluation protocol=Arora et al. [2024]2026.0524.86
WSDModel=1B, αpre=0.1, αmid=1.02026.0323.7
RESFORMERInput length=2k tokens, Evaluation protocol=Arora et al. [2024]2026.0522.8
Warmup-Stable-Only (WSO)Model=1B, αpre=1.0, αmid=1.02026.0322.7
DENSEFORMERInput length=2k tokens, Evaluation protocol=Arora et al. [2024]2026.0522.66
WSDModel=1B, αpre=1.0, αmid=0.02026.0322
WSDModel=1B, αpre=0.1, αmid=0.02026.0322
SATFORMERInput length=2k tokens, Evaluation protocol=Arora et al. [2024]2026.0521.41
MUDDFORMERInput length=2k tokens, Evaluation protocol=Arora et al. [2024]2026.0520.84
TRANSFORMERInput length=2k tokens, Evaluation protocol=Arora et al. [2024]2026.0520.07
CosineModel=1B, αpre=0.1, αmid=0.02026.0317.6
CosineModel=1B, αpre=0.1, αmid=1.02026.0317.1
LinearModel=1B, αpre=0.1, αmid=1.02026.0316.3
LinearModel=1B, αpre=0.1, αmid=0.02026.0316.2