Question Answering on ARC Challenge (ARC-Challenge Score)
37.3ARC-Challenge ScorePre-unlearn
Evaluation Results
| Method | Links | |
|---|---|---|
| Pre-unlearn2025.10 | 37.3 | |
| NPO w/ ZOAlgorithm=NPO, Optimizer=ZO2025.10 | 35.5 | |
| GradDiff w/ signAdamAlgorithm=GradDiff, Optimizer=signAdam2025.10 | 35.3 | |
| GradDiff w/ RSAlgorithm=GradDiff, Optimizer=RS2025.10 | 34.9 | |
| Echo-180MParams=180M, Tokens=10B, Zero-shot=true2026.05 | 34.2 | |
| GradDiffAlgorithm=GradDiff2025.10 | 33.6 | |
| GradDiff w/ SAMAlgorithm=GradDiff, Optimizer=SAM2025.10 | 33.5 | |
| GradDiff w/ signSGDAlgorithm=GradDiff, Optimizer=signSGD2025.10 | 33 | |
| NPO w/ signSGDAlgorithm=NPO, Optimizer=signSGD2025.10 | 31.7 | |
| NPO w/ signAdamAlgorithm=NPO, Optimizer=signAdam2025.10 | 28.8 | |
| NPO w/ RSAlgorithm=NPO, Optimizer=RS2025.10 | 28.2 | |
| GDN-180MParams=180M, Tokens=100B, Zero-shot=true2026.05 | 28.2 | |
| Transformer-180MParams=180M, Tokens=100B, Zero-shot=true2026.05 | 27.9 | |
| Mamba-3-SISO-180MParams=180M, Tokens=100B, Zero-shot=true2026.05 | 27.9 | |
| Mamba-3-MIMO-180MParams=180M, Tokens=100B, Zero-shot=true2026.05 | 27.7 | |
| GradDiff w/ ZOAlgorithm=GradDiff, Optimizer=ZO2025.10 | 27.4 | |
| NPOAlgorithm=NPO2025.10 | 27.3 | |
| Mamba-2-180MParams=180M, Tokens=100B, Zero-shot=true2026.05 | 27.3 | |
| NPO w/ SAMAlgorithm=NPO, Optimizer=SAM2025.10 | 26.7 | |
| Echo-50MParams=50M, Tokens=3B, Zero-shot=true2026.05 | 24.4 | |
| GradDiff w/ HybridAlgorithm=GradDiff, Optimizer=Hybrid2025.10 | 24.2 | |
| NPO w/ HybridAlgorithm=NPO, Optimizer=Hybrid2025.10 | 23.5 |