Question Answering and Reasoning on Macro-average (MMLU, MATH, GSM8K, BBH)
46Cost ReductionCISC
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CISCConfidence Method=P(True), Budget=102025.02 | 46 | 1.1 | |
| CISCConfidence Method=P(True), Budget=52025.02 | 41 | 1.6 | |
| CISCConfidence Method=Response Probability, Budget=102025.02 | 31 | 0.8 | |
| CISCConfidence Method=Verbal 1-100, Budget=102025.02 | 30 | 0.4 | |
| CISCConfidence Method=Verbal 1-100, Budget=52025.02 | 22 | 0.8 | |
| CISCConfidence Method=Response Probability, Budget=52025.02 | 22 | 1.1 | |
| CISCConfidence Method=Verbal Binary, Budget=52025.02 | 18 | 0.4 | |
| CISCConfidence Method=Verbal Binary, Budget=102025.02 | 10 | 0.2 |