General Reasoning on ARC, BBH, DROP, CommonsenseQA, SIQA
79CBRC ScoreARC
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| ARCPrompting=Standardized templates, Decoding=Greedy, Evaluation Metric=Exact match or execution-based2026.01 | 79 | 0.03 | 11 | 5,695 | 0.07 | 87 | 0.05 | 65 | |
| BBHPrompting=Standardized templates, Decoding=Greedy, Evaluation Metric=Exact match or execution-based2026.01 | 75 | 0.02 | 25 | 3,089 | 0.12 | 66 | 0.05 | 60 | |
| CommonsenseQAPrompting=Standardized templates, Decoding=Greedy, Evaluation Metric=Exact match or execution-based2026.01 | 75 | 0.07 | 17 | 3,785 | 0.11 | 57 | 0.02 | 54 | |
| SIQAPrompting=Standardized templates, Decoding=Greedy, Evaluation Metric=Exact match or execution-based2026.01 | 73 | 0.05 | 17 | 2,453 | 0.09 | 23 | 0.03 | 40 | |
| DROPPrompting=Standardized templates, Decoding=Greedy, Evaluation Metric=Exact match or execution-based2026.01 | 71 | 0.08 | 20 | 4,187 | 0.05 | 61 | 0.07 | 56 |