Algorithmic Reasoning on BBEH Mini
17.8AccuracyStatic
Evaluation results
| Method | Links | ||
|---|---|---|---|
| StaticBudget=40962026.04 | 17.8 | 12,428 | |
| TABBudget (B)=10k2026.04 | 17 | 5,236 | |
| StaticBudget=20482026.04 | 16.5 | 6,984 | |
| TABBudget (B)=8k2026.04 | 15.7 | 4,644 | |
| StaticBudget=10242026.04 | 15.4 | 4,394 | |
| TABBudget (B)=5k2026.04 | 15.4 | 3,867 | |
| StaticBudget=5122026.04 | 14.6 | 2,909 | |
| TABBudget (B)=3k2026.04 | 14.6 | 2,238 | |
| LLM-Judge IndividualSelection Strategy=Individual2026.04 | 13.9 | 2,152 | |
| LLM-Judge Multi-TurnSelection Strategy=Multi-Turn2026.04 | 13.9 | 1,949 | |
| StaticBudget=2562026.04 | 13.7 | 2,054 |