Closed-Book QA on ARC Challenge
63.7AccuracyFLAN 137B
Evaluation Results
| Method | Links | |
|---|---|---|
| FLAN 137BEvaluation protocol=few-shot, k=variable, Training strategy=leave-one-category-out2022.12 | 63.7 | |
| FLAN 137BEvaluation protocol=0-shot, Training strategy=leave-one-category-out2022.12 | 61.7 | |
| OPT-IML 175BEvaluation protocol=0-shot2022.12 | 61.2 | |
| OPT-IML 175BEvaluation protocol=few-shot, k=52022.12 | 58 | |
| OPT-IML 30BEvaluation protocol=0-shot2022.12 | 54.4 | |
| OPT-IML 30BEvaluation protocol=few-shot, k=52022.12 | 49.8 | |
| LaMDA-PT 137BEvaluation protocol=few-shot, k=variable2022.12 | 49.4 | |
| LaMDA-PT 137BEvaluation protocol=0-shot2022.12 | 42 | |
| OPT 175BEvaluation protocol=few-shot, k=52022.12 | 34.9 | |
| OPT 175BEvaluation protocol=0-shot2022.12 | 31 | |
| OPT 30BEvaluation protocol=few-shot, k=52022.12 | 29.6 | |
| OPT 30BEvaluation protocol=0-shot2022.12 | 27.4 |