Uncertainty Estimation on Average Across all subsets
77.3AUROCTop-5 + Tests + Pseudo
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Top-5 + Tests + PseudoModel=Qwen3-Coder-30B2026.06 | 77.3 | 80.3 | |
| Top-5 + Tests + PseudoModel=Qwen3-32B2026.06 | 74.7 | 79.2 | |
| Generated Tests (FUNC)Model=Qwen3-32B2026.06 | 74.5 | 75.9 | |
| Generated Tests (FUNC)Model=Qwen3-Coder-30B2026.06 | 73.8 | 77 | |
| Consistency (VR)Model=Qwen3-32B2026.06 | 70 | 76 | |
| Consistency (BLEU)Model=Qwen3-Coder-30B2026.06 | 69.2 | 73.1 | |
| Max EntropyModel=Qwen3-Coder-30B2026.06 | 68.8 | 73.4 | |
| Top-5 Entropy (LEX)Model=Qwen3-Coder-30B2026.06 | 68.8 | 75 | |
| Pseudo Consistency (ALGO)Model=Qwen3-Coder-30B2026.06 | 65.1 | 78.6 | |
| Consistency (BLEU)Model=Qwen3-32B2026.06 | 64.1 | 70.6 | |
| Pseudo Consistency (ALGO)Model=Qwen3-32B2026.06 | 63.6 | 69.3 | |
| Mean EntropyModel=Qwen3-Coder-30B2026.06 | 62.3 | 68.8 | |
| Symb. ClusteringModel=Qwen3-32B2026.06 | 62.2 | 70.9 | |
| Max ProbabilityModel=Qwen3-Coder-30B2026.06 | 62 | 69.6 | |
| Consistency (VR)Model=Qwen3-Coder-30B2026.06 | 61.6 | 68.5 | |
| Mean ProbabilityModel=Qwen3-Coder-30B2026.06 | 60.1 | 68.4 | |
| Top-5 Entropy (LEX)Model=Qwen3-32B2026.06 | 59.9 | 64.3 | |
| Symb. ClusteringModel=Qwen3-Coder-30B2026.06 | 56.8 | 67.7 | |
| Max ProbabilityModel=Qwen3-32B2026.06 | 52.9 | 62.2 | |
| Mean ProbabilityModel=Qwen3-32B2026.06 | 52.5 | 61.9 | |
| Mean EntropyModel=Qwen3-32B2026.06 | 50.4 | 58.4 | |
| Max EntropyModel=Qwen3-32B2026.06 | 50.2 | 59.1 |