Multi-task Language Understanding on MMLU (Comparable SC Samples, Acc Improvement)
47Comparable SC SamplesP(True)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| P(True)Confidence Method=P(True), Budget=102025.02 | 47 | 1 | |
| P(True)Confidence Method=P(True), Budget=52025.02 | 37 | 1.4 | |
| Verbal 1-100Confidence Method=Verbal 1-100, Budget=102025.02 | 32 | 0.7 | |
| Verbal 1-100Confidence Method=Verbal 1-100, Budget=52025.02 | 25 | 0.9 | |
| Response ProbabilityConfidence Method=Response Probability, Budget=102025.02 | 23 | 0.6 | |
| Verbal BinaryConfidence Method=Verbal Binary, Budget=52025.02 | 18 | 0.4 | |
| Response ProbabilityConfidence Method=Response Probability, Budget=52025.02 | 17 | 0.7 | |
| Verbal BinaryConfidence Method=Verbal Binary, Budget=102025.02 | 12 | 0.2 |