Reasoning on BBH (unseen)
42.38Total Average ScoreCOT-T5-11B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| COT-T5-11BZero-shot=true, Model Size=11B, CoT Fine-tuning=COT COLLECTION2023.05 | 42.38 | — | — | |
| FLAN-T5-11BZero-shot=true, Model Size=11B2023.05 | 39.78 | — | — | |
| T5-11B + COT FTZero-shot=true, Model Size=11B, CoT Fine-tuning=COT COLLECTION2023.05 | 39.54 | — | — | |
| GPT-3 (175B)Zero-shot=true, Model Size=175B2023.05 | 38.3 | — | — | |
| COT-T5-3BZero-shot=true, Model Size=3B, CoT Fine-tuning=COT COLLECTION2023.05 | 37.29 | — | — | |
| T5-3B + COT FTZero-shot=true, Model Size=3B, CoT Fine-tuning=COT COLLECTION2023.05 | 36.74 | — | — | |
| FLAN-T5-3BZero-shot=true, Model Size=3B2023.05 | 35.6 | — | — | |
| T0-11BZero-shot=true, Model Size=11B2023.05 | 32.7 | — | — | |
| TK-INSTRUCT-11BZero-shot=true, Model Size=11B2023.05 | 32.16 | — | — | |
| TK-INSTRUCT-3BZero-shot=true, Model Size=3B2023.05 | 29.88 | — | — | |
| T0-3BZero-shot=true, Model Size=3B2023.05 | 27.05 | — | — | |
| T5-LM-3BZero-shot=true, Model Size=3B2023.05 | 26.82 | — | — | |
| Best-of-16Strategy=Best-of-16, Inference Budget=C=162026.03 | — | 78.2 | — | |
| CoT2-MetaStrategy=Ours (CoT2-Meta), Inference Budget=C=162026.03 | — | 84.5 | — | |
| Greedy CoTStrategy=Greedy CoT, Inference Budget=C=162026.03 | — | 74.5 | — | |
| ReST-MCTS*Strategy=ReST-MCTS*, Inference Budget=C=162026.03 | — | 82.1 | — | |
| Vanilla ToTStrategy=Vanilla ToT, Inference Budget=C=162026.03 | — | 80.5 | — |