List your company
ResearchFormerly WizWand

Uncertainty Estimation on MBPP

0.791AUROCTop-5 + Tests + Pseudo
0.498760.574630.65050.72637Jun 8, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Top-5 + Tests + PseudoModel=Devstral-Small-25052026.060.7910.785
Top-5 + Tests + PseudoBackbone=Devstral-Small-25052026.060.7910.785
Generated Tests (FUNC)Model=Devstral-Small-25052026.060.7760.735
Generated Tests (FUNC)Backbone=Devstral-Small-25052026.060.7760.735
Top-5 + Tests + PseudoModel=Qwen3-Coder-30B2026.060.7480.767
Top-5 + Tests + PseudoModel=Qwen3-14B2026.060.7460.761
Top-5 + Tests + PseudoBackbone=Qwen3-14B2026.060.7460.761
Generated Tests (FUNC)Model=DeepSeek-Coder-V22026.060.7410.736
Generated Tests (FUNC)Backbone=DeepSeek-Coder-V22026.060.7410.736
Top-5 + Tests + PseudoModel=DeepSeek-Coder-V22026.060.740.755
Top-5 + Tests + PseudoBackbone=DeepSeek-Coder-V22026.060.740.755
Generated Tests (FUNC)Model=Qwen3-Coder-30B2026.060.7310.722
Generated Tests (FUNC)Model=Qwen3-14B2026.060.730.702
Generated Tests (FUNC)Backbone=Qwen3-14B2026.060.730.702
Top-5 + Tests + PseudoModel=Qwen3-32B2026.060.7270.738
Pseudo Consistency (ALGO)Model=DeepSeek-Coder-V22026.060.7220.636
Pseudo Consistency (ALGO)Backbone=DeepSeek-Coder-V22026.060.7220.636
Generated Tests (FUNC)Model=Qwen3-32B2026.060.7110.724
Consistency (VR)Model=Qwen3-32B2026.060.6630.713
Symb. ClusteringModel=Qwen3-14B2026.060.6610.691
Symb. ClusteringBackbone=Qwen3-14B2026.060.6610.691
Consistency (VR)Model=Qwen3-14B2026.060.6480.684
Consistency (VR)Backbone=Qwen3-14B2026.060.6480.684
Pseudo Consistency (ALGO)Model=Devstral-Small-25052026.060.6470.62
Pseudo Consistency (ALGO)Backbone=Devstral-Small-25052026.060.6470.62
Consistency (BLEU)Model=Qwen3-14B2026.060.640.673
Consistency (BLEU)Backbone=Qwen3-14B2026.060.640.673
Max ProbabilityModel=DeepSeek-Coder-V22026.060.6170.634
Consistency (VR)Model=DeepSeek-Coder-V22026.060.6160.639
Top-5 Entropy (LEX)Model=DeepSeek-Coder-V22026.060.6160.653
Consistency (VR)Backbone=DeepSeek-Coder-V22026.060.6160.639
Top-5 Entropy (LEX)Backbone=DeepSeek-Coder-V22026.060.6160.653
Pseudo Consistency (ALGO)Model=Qwen3-14B2026.060.6150.65
Pseudo Consistency (ALGO)Backbone=Qwen3-14B2026.060.6150.65
Top-5 Entropy (LEX)Model=Devstral-Small-25052026.060.6140.653
Top-5 Entropy (LEX)Backbone=Devstral-Small-25052026.060.6140.653
Consistency (VR)Model=Devstral-Small-25052026.060.6110.642
Consistency (VR)Backbone=Devstral-Small-25052026.060.6110.642
Pseudo Consistency (ALGO)Model=Qwen3-32B2026.060.6110.656
Consistency (BLEU)Model=Qwen3-32B2026.060.6030.664
Pseudo Consistency (ALGO)Model=Qwen3-Coder-30B2026.060.6020.66
Max EntropyModel=DeepSeek-Coder-V22026.060.6010.62
Consistency (BLEU)Model=Qwen3-Coder-30B2026.060.60.619
Top-5 Entropy (LEX)Model=Qwen3-Coder-30B2026.060.5990.629
Consistency (BLEU)Model=Devstral-Small-25052026.060.5970.621
Consistency (BLEU)Backbone=Devstral-Small-25052026.060.5970.621
Mean ProbabilityModel=DeepSeek-Coder-V22026.060.5960.612
Consistency (BLEU)Model=DeepSeek-Coder-V22026.060.5960.641
Consistency (BLEU)Backbone=DeepSeek-Coder-V22026.060.5960.641
Top-5 Entropy (LEX)Model=Qwen3-32B2026.060.5920.657
Symb. ClusteringModel=Qwen3-Coder-30B2026.060.5920.616
Mean EntropyModel=Qwen3-Coder-30B2026.060.5850.627
Mean ProbabilityModel=Qwen3-32B2026.060.580.658
Max ProbabilityModel=Qwen3-32B2026.060.5730.66
Top-5 Entropy (LEX)Model=Qwen3-14B2026.060.570.616
Top-5 Entropy (LEX)Backbone=Qwen3-14B2026.060.570.616
Max EntropyModel=Qwen3-Coder-30B2026.060.570.607
Symb. ClusteringModel=DeepSeek-Coder-V22026.060.5650.639
Symb. ClusteringBackbone=DeepSeek-Coder-V22026.060.5650.639
Symb. ClusteringModel=Qwen3-32B2026.060.560.66
Mean EntropyModel=DeepSeek-Coder-V22026.060.5440.611
Mean EntropyBackbone=DeepSeek-Coder-V22026.060.5440.611
Mean EntropyModel=Qwen3-32B2026.060.5440.617
Max ProbabilityModel=Qwen3-Coder-30B2026.060.5360.609
Max ProbabilityModel=Devstral-Small-25052026.060.5350.57
Max EntropyModel=Qwen3-14B2026.060.5320.569
Max EntropyModel=Devstral-Small-25052026.060.5320.563
Max EntropyModel=Qwen3-32B2026.060.5320.632
Mean ProbabilityModel=Devstral-Small-25052026.060.5290.561
Mean EntropyModel=Devstral-Small-25052026.060.5280.558
Mean EntropyBackbone=Devstral-Small-25052026.060.5280.558
Consistency (VR)Model=Qwen3-Coder-30B2026.060.5280.56
Mean EntropyModel=Qwen3-14B2026.060.5260.565
Mean EntropyBackbone=Qwen3-14B2026.060.5260.565
Max ProbabilityModel=Qwen3-14B2026.060.5210.564
Mean ProbabilityModel=Qwen3-14B2026.060.5180.555
Symb. ClusteringModel=Devstral-Small-25052026.060.5140.575
Symb. ClusteringBackbone=Devstral-Small-25052026.060.5140.575
Mean ProbabilityModel=Qwen3-Coder-30B2026.060.510.61