Reasoning Question Answering on ARC
94.62AccuracyIoT
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| IoTModel=GPT-4o mini2026.03 | 94.62 | — | — | — | — | — | — | — | |
| CoTModel=GPT-4o mini2026.03 | 94.54 | — | — | — | — | — | — | — | |
| IoTModel=Olmo-2-13B2026.03 | 88.65 | — | — | — | — | — | — | — | |
| IoTModel=Olmo-2-7B2026.03 | 87.54 | — | — | — | — | — | — | — | |
| SCModel=Olmo-2-13B2026.03 | 86.01 | — | — | — | — | — | — | — | |
| EoTModel=Olmo-2-13B2026.03 | 85.58 | — | — | — | — | — | — | — | |
| CoTModel=Olmo-2-13B2026.03 | 85.32 | — | — | — | — | — | — | — | |
| CoTModel=Olmo-2-7B2026.03 | 84.21 | — | — | — | — | — | — | — | |
| IoTModel=Llama-3.3-8B2026.03 | 83.79 | — | — | — | — | — | — | — | |
| SCModel=Llama-3.3-8B2026.03 | 82.57 | — | — | — | — | — | — | — | |
| EoTModel=Llama-3.3-8B2026.03 | 81.99 | — | — | — | — | — | — | — | |
| CoTModel=Llama-3.3-8B2026.03 | 81.06 | — | — | — | — | — | — | — | |
| SwiftMode=Thinking Mode2025.06 | 79.93 | 97.9 | 7.19 | 8.12 | 7.46 | 6.6 | 6.95 | 7.75 | |
| EoTModel=Olmo-2-7B2026.03 | 79.69 | — | — | — | — | — | — | — | |
| SCModel=Olmo-2-7B2026.03 | 79.22 | — | — | — | — | — | — | — | |
| RefitMode=Thinking Mode2025.06 | 78.89 | 97.9 | 7.12 | 8.03 | 7.37 | 6.56 | 6.88 | 7.66 | |
| STaR-GATE-DMode=Thinking Mode2025.06 | 75.78 | 23.9 | 5.47 | 6.99 | 5.65 | 4.83 | 4.74 | 5.95 | |
| STaR-GATEMode=Thinking Mode2025.06 | 69.9 | 90 | 6.67 | 7.48 | 6.94 | 6.22 | 6.5 | 7.11 | |
| DPOMode=Thinking Mode2025.06 | 64.71 | 8.7 | 5.72 | 6.84 | 6.05 | 5.3 | 5.21 | 6.02 | |
| StepDPOMode=Thinking Mode2025.06 | 64.01 | 8 | 5.46 | 6.6 | 5.76 | 5.04 | 4.88 | 5.83 | |
| BaseMode=Thinking Mode2025.06 | 37.72 | 75.1 | 6.47 | 7.32 | 6.56 | 5.8 | 6.4 | 6.92 |