Reasoning on BBEH (Accuracy, Delta Avg)
81.2AccuracyCoT2-Meta
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CoT2-MetaBackbone=DeepSeek-V3.2, Strategy=Ours (CoT2-Meta)2026.03 | 81.2 | 10.5 | |
| Vanilla ToTBackbone=DeepSeek-V3.2, Strategy=Vanilla ToT2026.03 | 77.8 | 5.7 | |
| Best-of-16Backbone=DeepSeek-V3.2, Strategy=Best-of-162026.03 | 75.9 | 3 | |
| CoT2-MetaBackbone=Claude-4.5, Strategy=Ours (CoT2-Meta)2026.03 | 75.8 | 14.5 | |
| Greedy CoTBackbone=DeepSeek-V3.2, Strategy=Greedy CoT2026.03 | 74.4 | — | |
| CoT2-MetaBackbone=Qwen2.5-VL-7B, Strategy=Ours (CoT2-Meta)2026.03 | 72.5 | 12.2 | |
| Vanilla ToTBackbone=Claude-4.5, Strategy=Vanilla ToT2026.03 | 71.2 | 8.3 | |
| Vanilla ToTBackbone=Qwen2.5-VL-7B, Strategy=Vanilla ToT2026.03 | 69.1 | 6.4 | |
| Best-of-16Backbone=Claude-4.5, Strategy=Best-of-162026.03 | 68.9 | 4.8 | |
| Best-of-16Backbone=Qwen2.5-VL-7B, Strategy=Best-of-162026.03 | 66.8 | 3.4 | |
| Greedy CoTBackbone=Claude-4.5, Strategy=Greedy CoT2026.03 | 65.4 | — | |
| Greedy CoTBackbone=Qwen2.5-VL-7B, Strategy=Greedy CoT2026.03 | 64.5 | — |