Scientific Reasoning on ARC-C (0-shot, test)
83.87Pass@1 AccHEX
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HEXGen. Len.=256, Training Necessity=Training-free, Setting=0-shot, reimplementation=true2026.02 | 83.87 | 3,200 | |
| dVOTINGGen. Len.=256, Training Necessity=Training-free, Setting=0-shot2026.02 | 83.36 | 415.6 | |
| HEXGen. Len.=128, Training Necessity=Training-free, Setting=0-shot, reimplementation=true2026.02 | 82.67 | 1,600 | |
| Majority VotingGen. Len.=256, Training Necessity=Training-free, Setting=0-shot2026.02 | 81.31 | 640 | |
| dVOTINGGen. Len.=128, Training Necessity=Training-free, Setting=0-shot2026.02 | 80.97 | 265.4 | |
| Majority VotingGen. Len.=128, Training Necessity=Training-free, Setting=0-shot2026.02 | 77.65 | 320 | |
| LLaDA-8B-InstructGen. Len.=256, Training Necessity=Pre-training, Setting=0-shot2026.02 | 74.66 | 256 | |
| LLaDA-8B-InstructGen. Len.=128, Training Necessity=Pre-training, Setting=0-shot2026.02 | 66.13 | 128 |