Multiple Choice Question Answering on MMLU-Pro Law (Uncertainty Quantification)
0.997Calibration Threshold (q-hat)Conformal Social Choice
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Conformal Social ChoiceDebate Round=3, Alpha=0.05, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.997 | 96.4 | 6.91 | 6.2 | 75 | |
| Conformal Social ChoiceDebate Round=2, Alpha=0.05, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.994 | 96.5 | 6.79 | 5.5 | 62.5 | |
| Conformal Social ChoiceDebate Round=1, Alpha=0.05, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.99 | 96.2 | 6.49 | 4 | 100 | |
| Conformal Social ChoiceDebate Round=3, Alpha=0.10, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.984 | 90.7 | 3.94 | 20.9 | 92.3 | |
| Conformal Social ChoiceDebate Round=0, Alpha=0.05, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.98 | 97.6 | 6.99 | 1.5 | 100 | |
| Conformal Social ChoiceDebate Round=2, Alpha=0.10, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.977 | 90 | 3.65 | 18.5 | 69.6 | |
| Conformal Social ChoiceDebate Round=1, Alpha=0.10, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.967 | 88.9 | 3.65 | 14.3 | 81.8 | |
| Conformal Social ChoiceDebate Round=0, Alpha=0.10, Model Ensemble=Haiku + DeepSeek-R1 + Qwen-3 32B2026.04 | 0.945 | 89.5 | 3.76 | 6.4 | 91.4 |