List your company
ResearchFormerly WizWand

Code Correctness Classification on LiveSQLBench SQLite

0.842AUROCFunctional Sets Confidence
0.5040.591750.67950.76725May 27, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Functional Sets ConfidenceModel=GPT-4o-mini2026.050.842
Functional EntropyModel=GPT-4o-mini2026.050.84
P(True)Model=GPT-4o2026.050.818
Functional Equivalence RateModel=GPT-4o-mini2026.050.782
Functional Sets ConfidenceModel=GPT-4o2026.050.778
Functional EntropyModel=GPT-4o2026.050.775
Functional Sets ConfidenceModel=Gemini-2.5-Pro2026.050.75
Functional Equivalence RateModel=GPT-4o2026.050.749
P(True)Model=Gemini-2.5-Flash-Lite2026.050.749
Functional EntropyModel=Gemini-2.5-Pro2026.050.745
Functional EntropyModel=Gemini-2.5-Flash-Lite2026.050.745
Functional Sets ConfidenceModel=Gemini-2.5-Flash-Lite2026.050.743
Verbalized ConfidenceModel=Gemini-2.5-Flash-Lite2026.050.739
Functional Equivalence RateModel=Gemini-2.5-Flash-Lite2026.050.738
Verbalized ConfidenceModel=GPT-4o-mini2026.050.737
Cosine SimilarityModel=Gemini-2.5-Pro2026.050.726
Cosine SimilarityModel=GPT-4o2026.050.721
Minimum Token NegentropyModel=Gemini-2.5-Flash-Lite2026.050.712
Minimum Token NegentropyModel=GPT-4o2026.050.706
Cosine SimilarityModel=GPT-4o-mini2026.050.705
Functional Equivalence RateModel=Gemini-2.5-Pro2026.050.702
P(True)Model=Gemini-2.5-Pro2026.050.695
Average Token NegentropyModel=GPT-4o-mini2026.050.694
Minimum Token NegentropyModel=GPT-4o-mini2026.050.687
P(True)Model=GPT-4o-mini2026.050.687
Cosine SimilarityModel=Gemini-2.5-Flash-Lite2026.050.683
Minimum ProbabilityModel=GPT-4o2026.050.679
Sequence ProbabilityModel=GPT-4o-mini2026.050.678
Functional Sets ConfidenceModel=Gemini-2.5-Flash2026.050.678
Functional EntropyModel=Gemini-2.5-Flash2026.050.674
Probability MarginModel=GPT-4o-mini2026.050.673
Minimum ProbabilityModel=GPT-4o-mini2026.050.672
Cosine SimilarityModel=Gemini-2.5-Flash2026.050.665
Average Token NegentropyModel=Gemini-2.5-Flash-Lite2026.050.662
Sequence ProbabilityModel=Gemini-2.5-Flash-Lite2026.050.658
Functional Equivalence RateModel=Gemini-2.5-Flash2026.050.657
Minimum Token NegentropyModel=Gemini-2.5-Flash2026.050.654
P(True)Model=Gemini-2.5-Flash2026.050.647
Sequence ProbabilityModel=GPT-4o2026.050.646
Minimum ProbabilityModel=Gemini-2.5-Flash-Lite2026.050.641
Average Token NegentropyModel=GPT-4o2026.050.632
Probability MarginModel=Gemini-2.5-Flash-Lite2026.050.629
Minimum ProbabilityModel=Gemini-2.5-Flash2026.050.628
Minimum Token NegentropyModel=Gemini-2.5-Pro2026.050.623
Probability MarginModel=GPT-4o2026.050.616
Minimum ProbabilityModel=Gemini-2.5-Pro2026.050.602
Verbalized ConfidenceModel=Gemini-2.5-Pro2026.050.596
Verbalized ConfidenceModel=GPT-4o2026.050.596
Average Token NegentropyModel=Gemini-2.5-Flash2026.050.558
Sequence ProbabilityModel=Gemini-2.5-Flash2026.050.554
Probability MarginModel=Gemini-2.5-Flash2026.050.546
Verbalized ConfidenceModel=Gemini-2.5-Flash2026.050.535
Sequence ProbabilityModel=Gemini-2.5-Pro2026.050.529
Average Token NegentropyModel=Gemini-2.5-Pro2026.050.528
Probability MarginModel=Gemini-2.5-Pro2026.050.517