Uncertainty Quantification on NQ
85.1AUROCClustered Self-Assessment
Evaluation results
| Method | Links | |
|---|---|---|
| Clustered Self-AssessmentModel=Qwen3-14B, Number of additional samples=22026.06 | 85.1 | |
| Clustered Self-AssessmentModel=Qwen2.5-32B, Number of additional samples=22026.06 | 84 | |
| Clustered Self-AssessmentModel=Qwen2.5-7B, Number of additional samples=22026.06 | 83.9 | |
| Clustered Self-AssessmentModel=Gemma-3-12B, Number of additional samples=22026.06 | 83.7 | |
| Clustered Self-AssessmentModel=Qwen3-30B-A3B, Number of additional samples=22026.06 | 83.6 | |
| Clustered Self-AssessmentModel=Qwen2.5-14B, Number of additional samples=22026.06 | 83.3 | |
| Clustered Self-AssessmentModel=Gemma-3-27B, Number of additional samples=22026.06 | 82.3 | |
| P(True)Model=Qwen3-14B, Number of additional samples=22026.06 | 79.9 | |
| DegModel=Gemma-3-12B, Number of additional samples=22026.06 | 79 | |
| P(True)Model=Qwen2.5-32B, Number of additional samples=22026.06 | 78.5 | |
| P(True)Model=Qwen3-30B-A3B, Number of additional samples=22026.06 | 78.3 | |
| EigVModel=Qwen2.5-7B, Number of additional samples=22026.06 | 78.1 | |
| EigVModel=Qwen3-14B, Number of additional samples=22026.06 | 78 | |
| DegModel=Gemma-3-27B, Number of additional samples=22026.06 | 77.7 | |
| ProbabilityModel=Qwen2.5-7B, Number of additional samples=22026.06 | 77.6 | |
| sentenceSARModel=Qwen2.5-7B, Number of additional samples=22026.06 | 77.6 | |
| SARModel=Qwen2.5-7B, Number of additional samples=22026.06 | 77.6 | |
| ProbabilityModel=Qwen3-14B, Number of additional samples=22026.06 | 77.4 | |
| Predictive EntropyModel=Qwen2.5-7B, Number of additional samples=22026.06 | 77 | |
| SARModel=Qwen3-14B, Number of additional samples=22026.06 | 77 | |
| sentenceSARModel=Qwen3-14B, Number of additional samples=22026.06 | 76.8 | |
| EigVModel=Qwen2.5-32B, Number of additional samples=22026.06 | 76.8 | |
| SARModel=Gemma-3-12B, Number of additional samples=22026.06 | 76.5 | |
| ProbabilityModel=Qwen2.5-32B, Number of additional samples=22026.06 | 76.3 | |
| DegModel=Qwen3-14B, Number of additional samples=22026.06 | 76.3 | |
| Predictive EntropyModel=Qwen3-14B, Number of additional samples=22026.06 | 76.2 | |
| EigVModel=Qwen3-30B-A3B, Number of additional samples=22026.06 | 76 | |
| sentenceSARModel=Qwen2.5-32B, Number of additional samples=22026.06 | 75.9 | |
| EigVModel=Gemma-3-12B, Number of additional samples=22026.06 | 75.9 | |
| DegModel=Qwen2.5-7B, Number of additional samples=22026.06 | 75.8 | |
| ProbabilityModel=Gemma-3-27B, Number of additional samples=22026.06 | 75.5 | |
| SARModel=Qwen2.5-32B, Number of additional samples=22026.06 | 75.5 | |
| tokenSARModel=Gemma-3-12B, Number of additional samples=22026.06 | 75.4 | |
| SARModel=Qwen3-30B-A3B, Number of additional samples=22026.06 | 75.4 | |
| Predictive EntropyModel=Qwen2.5-32B, Number of additional samples=22026.06 | 75.3 | |
| Semantic EntropyModel=Qwen2.5-7B, Number of additional samples=22026.06 | 75.3 | |
| EigVModel=Qwen2.5-14B, Number of additional samples=22026.06 | 75.3 | |
| LN EntropyModel=Qwen2.5-7B, Number of additional samples=22026.06 | 75.1 | |
| tokenSARModel=Gemma-3-27B, Number of additional samples=22026.06 | 75 | |
| Semantic EntropyModel=Qwen3-14B, Number of additional samples=22026.06 | 75 | |
| EigVModel=Gemma-3-27B, Number of additional samples=22026.06 | 75 | |
| PerplexityModel=Gemma-3-27B, Number of additional samples=22026.06 | 74.9 | |
| PerplexityModel=Qwen2.5-7B, Number of additional samples=22026.06 | 74.8 | |
| ProbabilityModel=Qwen3-30B-A3B, Number of additional samples=22026.06 | 74.8 | |
| DegModel=Qwen2.5-32B, Number of additional samples=22026.06 | 74.8 | |
| ProbabilityModel=Qwen2.5-14B, Number of additional samples=22026.06 | 74.7 | |
| sentenceSARModel=Qwen2.5-14B, Number of additional samples=22026.06 | 74.7 | |
| sentenceSARModel=Qwen3-30B-A3B, Number of additional samples=22026.06 | 74.7 | |
| PerplexityModel=Gemma-3-12B, Number of additional samples=22026.06 | 74.5 | |
| SARModel=Qwen2.5-14B, Number of additional samples=22026.06 | 74.5 | |
| LN EntropyModel=Qwen3-14B, Number of additional samples=22026.06 | 74.4 | |
| Predictive EntropyModel=Qwen2.5-14B, Number of additional samples=22026.06 | 74.2 | |
| Predictive EntropyModel=Qwen3-30B-A3B, Number of additional samples=22026.06 | 74.2 | |
| PerplexityModel=Qwen3-14B, Number of additional samples=22026.06 | 73.7 | |
| DegModel=Qwen3-30B-A3B, Number of additional samples=22026.06 | 73.7 | |
| Semantic EntropyModel=Qwen2.5-32B, Number of additional samples=22026.06 | 73.5 | |
| LN EntropyModel=Qwen3-30B-A3B, Number of additional samples=22026.06 | 73.4 | |
| DegModel=Qwen2.5-14B, Number of additional samples=22026.06 | 73.3 | |
| tokenSARModel=Qwen2.5-7B, Number of additional samples=22026.06 | 73.2 | |
| sentenceSARModel=Gemma-3-12B, Number of additional samples=22026.06 | 73.2 | |
| P(True)Model=Qwen2.5-14B, Number of additional samples=22026.06 | 73.1 | |
| LN EntropyModel=Qwen2.5-32B, Number of additional samples=22026.06 | 73 | |
| PerplexityModel=Qwen3-30B-A3B, Number of additional samples=22026.06 | 72.9 | |
| ProbabilityModel=Gemma-3-12B, Number of additional samples=22026.06 | 72.9 | |
| P(True)Model=Qwen2.5-7B, Number of additional samples=22026.06 | 72.9 | |
| SARModel=Gemma-3-27B, Number of additional samples=22026.06 | 72.9 | |
| PerplexityModel=Qwen2.5-32B, Number of additional samples=22026.06 | 72.8 | |
| tokenSARModel=Qwen3-14B, Number of additional samples=22026.06 | 72.5 | |
| Semantic EntropyModel=Gemma-3-27B, Number of additional samples=22026.06 | 72.5 | |
| Semantic EntropyModel=Qwen3-30B-A3B, Number of additional samples=22026.06 | 72.4 | |
| Predictive EntropyModel=Gemma-3-12B, Number of additional samples=22026.06 | 72.2 | |
| sentenceSARModel=Gemma-3-27B, Number of additional samples=22026.06 | 72.1 | |
| LN EntropyModel=Gemma-3-12B, Number of additional samples=22026.06 | 72 | |
| Predictive EntropyModel=Gemma-3-27B, Number of additional samples=22026.06 | 71.9 | |
| Semantic EntropyModel=Qwen2.5-14B, Number of additional samples=22026.06 | 71.9 | |
| LN EntropyModel=Qwen2.5-14B, Number of additional samples=22026.06 | 71.6 | |
| tokenSARModel=Qwen2.5-32B, Number of additional samples=22026.06 | 71.4 | |
| tokenSARModel=Qwen3-30B-A3B, Number of additional samples=22026.06 | 71.4 | |
| EccModel=Qwen3-14B, Number of additional samples=22026.06 | 71.4 | |
| Eigen ScoreModel=Qwen2.5-7B, Number of additional samples=22026.06 | 71.3 | |
| Semantic EntropyModel=Gemma-3-12B, Number of additional samples=22026.06 | 70.7 | |
| EccModel=Qwen2.5-7B, Number of additional samples=22026.06 | 70.7 | |
| PerplexityModel=Qwen2.5-14B, Number of additional samples=22026.06 | 70.6 | |
| EccModel=Qwen2.5-32B, Number of additional samples=22026.06 | 70.2 | |
| tokenSARModel=Qwen2.5-14B, Number of additional samples=22026.06 | 70.1 | |
| Eigen ScoreModel=Qwen3-14B, Number of additional samples=22026.06 | 69.9 | |
| EccModel=Qwen2.5-14B, Number of additional samples=22026.06 | 69.7 | |
| LN EntropyModel=Gemma-3-27B, Number of additional samples=22026.06 | 69.3 | |
| EccModel=Qwen3-30B-A3B, Number of additional samples=22026.06 | 69.2 | |
| Eigen ScoreModel=Qwen2.5-32B, Number of additional samples=22026.06 | 69.1 | |
| Eigen ScoreModel=Qwen2.5-14B, Number of additional samples=22026.06 | 67.8 | |
| EccModel=Gemma-3-27B, Number of additional samples=22026.06 | 67 | |
| Eigen ScoreModel=Qwen3-30B-A3B, Number of additional samples=22026.06 | 66.7 | |
| EccModel=Gemma-3-12B, Number of additional samples=22026.06 | 66.5 | |
| P(True)Model=Gemma-3-27B, Number of additional samples=22026.06 | 65.9 | |
| Eigen ScoreModel=Gemma-3-27B, Number of additional samples=22026.06 | 64.1 | |
| Eigen ScoreModel=Gemma-3-12B, Number of additional samples=22026.06 | 63.1 | |
| P(True)Model=Gemma-3-12B, Number of additional samples=22026.06 | 60.9 |