Complex Reasoning on BIG-bench Hard
39.3Orig ScoreFLAN-T5
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FLAN-T5#Examples=14,336,000, Evaluation Protocol=Prior Work2022.12 | 39.3 | 40 | |
| T5-LM on Unnatural Instructions + Instruction Paraphrases#Examples=240,670, Evaluation Protocol=Our Approach2022.12 | 28.1 | 29.4 | |
| T0++#Examples=12,492,800, Evaluation Protocol=Prior Work2022.12 | 20.2 | 13.9 | |
| T5-LM on Unnatural Instructions#Examples=64,000, Evaluation Protocol=Our Approach2022.12 | 16 | 29.5 | |
| T5-LM on Super-Natural Instructions#Examples=64,000, Evaluation Protocol=Direct Comparison Baseline2022.12 | 10.2 | 29.7 | |
| Tk-Instruct#Examples=75,417, Evaluation Protocol=Prior Work2022.12 | 5.8 | 11.8 | |
| T5-LM#Examples=0, Evaluation Protocol=Prior Work2022.12 | 0 | 0.7 |