Downstream Task Evaluation on LAMBADA, ARC, WinoGrande, PIQA, HellaSwag, SciQ, and RACE
45LAMBADA (OpenAI)PonderLM-2
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| PonderLM-2Shots=0-shot, Inf. FLOPs (G/token)=9.84, Pondering Config=3 steps2026.03 | 45 | 47.6 | 36.6 | 21.9 | 51.6 | 64.4 | 32.3 | 80.6 | 31.4 | 45.7 | |
| PonderLM-3Shots=0-shot, Inf. FLOPs (G/token)=8.86, Pondering Config=max 3 steps2026.03 | 43.3 | 48.7 | 35 | 21.9 | 50.7 | 64.2 | 32.2 | 80 | 29.3 | 45 | |
| PonderLM-3Shots=5-shot, Inf. FLOPs (G/token)=8.86, Pondering Config=max 3 steps2026.03 | 40.8 | 50.7 | 36 | 22 | 52.6 | 64.9 | 32.3 | 86.9 | 31.8 | 46.4 | |
| LoopedLMShots=0-shot, Inf. FLOPs (G/token)=9.84, Pondering Config=4 loops2026.03 | 40.4 | 47.2 | 28.6 | 20.4 | 50.8 | 64.4 | 30.6 | 75.4 | 28.8 | 43 | |
| PonderLM-1Shots=0-shot, Inf. FLOPs (G/token)=9.84, Pondering Config=3 steps2026.03 | 39.6 | 47.8 | 29.3 | 20.4 | 50 | 64.1 | 30.4 | 77.2 | 29.9 | 43.2 | |
| PauseShots=0-shot, Inf. FLOPs (G/token)=9.84, Pondering Config=3 pauses2026.03 | 38.1 | 47.5 | 27.8 | 20.3 | 50.3 | 63.8 | 29.8 | 76.3 | 29 | 42.5 | |
| PonderLM-2Shots=5-shot, Inf. FLOPs (G/token)=9.84, Pondering Config=3 steps2026.03 | 37.6 | 50.6 | 34 | 21.8 | 53.5 | 65 | 32.5 | 87.5 | 31.2 | 46 | |
| MoRShots=0-shot, Inf. FLOPs (G/token)=8.44, Pondering Config=max 3 steps2026.03 | 37 | 45.3 | 26.6 | 19.9 | 50.1 | 63.8 | 30 | 77.3 | 28.6 | 42 | |
| LoopedLMShots=5-shot, Inf. FLOPs (G/token)=9.84, Pondering Config=4 loops2026.03 | 34.2 | 48.2 | 26.6 | 20.6 | 47 | 64.2 | 30.7 | 83.4 | 27.8 | 42.5 | |
| PonderLM-1Shots=5-shot, Inf. FLOPs (G/token)=9.84, Pondering Config=3 steps2026.03 | 34 | 49.1 | 28.4 | 21.8 | 49.7 | 64.1 | 30.9 | 84.4 | 29.5 | 43.5 | |
| MoRShots=5-shot, Inf. FLOPs (G/token)=8.44, Pondering Config=max 3 steps2026.03 | 31.5 | 48.5 | 25.3 | 20 | 51 | 63.7 | 30.6 | 84.7 | 28 | 42.6 | |
| PauseShots=5-shot, Inf. FLOPs (G/token)=9.84, Pondering Config=3 pauses2026.03 | 30.7 | 46.9 | 26.3 | 21.8 | 51.6 | 63.4 | 30.1 | 81.8 | 28.5 | 42.3 |