Multiple Choice Question Answering on MMLU-Pro Psychology (Uncertainty Quantification)
0.983Calibration Threshold (q-hat)Conformal Social Choice
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Conformal Social ChoiceDebate Round=3, Alpha=0.05, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.983 | 93.5 | 2.45 | 54.1 | 100 | |
| Conformal Social ChoiceDebate Round=2, Alpha=0.05, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.978 | 93 | 2.27 | 52.4 | 90.9 | |
| Conformal Social ChoiceDebate Round=1, Alpha=0.05, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.967 | 93.5 | 2.23 | 49.6 | 87.5 | |
| Conformal Social ChoiceDebate Round=0, Alpha=0.05, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.954 | 94.7 | 2.59 | 39.6 | 96.8 | |
| Conformal Social ChoiceDebate Round=3, Alpha=0.10, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.95 | 88.7 | 1.44 | 76.2 | 83.3 | |
| Conformal Social ChoiceDebate Round=2, Alpha=0.10, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.944 | 88.5 | 1.44 | 74.7 | 93.3 | |
| Conformal Social ChoiceDebate Round=1, Alpha=0.10, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.933 | 88.5 | 1.49 | 70.9 | 84.8 | |
| Conformal Social ChoiceDebate Round=0, Alpha=0.10, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.89 | 88.7 | 1.5 | 62.7 | 92.4 |