Multiple Choice Question Answering on MMLU-Pro Math (Calibration & Set Prediction)
0.126Calibration Threshold (q-hat)Conformal Social Choice
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Conformal Social ChoiceDebate Round=3, Alpha=0.10, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.126 | 93.2 | 0.98 | 97.8 | 100 | |
| Conformal Social ChoiceDebate Round=2, Alpha=0.10, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.229 | 92.9 | 0.97 | 97.5 | 69.2 | |
| Conformal Social ChoiceDebate Round=1, Alpha=0.10, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.359 | 91.6 | 0.96 | 95.6 | 87.5 | |
| Conformal Social ChoiceDebate Round=0, Alpha=0.10, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.605 | 88.6 | 0.92 | 92 | 96.1 | |
| Conformal Social ChoiceDebate Round=2, Alpha=0.05, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.704 | 94.5 | 1.02 | 97.6 | 83.3 | |
| Conformal Social ChoiceDebate Round=1, Alpha=0.05, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.718 | 94.8 | 1.06 | 94.1 | 89.2 | |
| Conformal Social ChoiceDebate Round=3, Alpha=0.05, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.74 | 94.5 | 1.01 | 98.2 | 100 | |
| Conformal Social ChoiceDebate Round=0, Alpha=0.05, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.75 | 94.5 | 1.26 | 74.9 | 96.4 |