Code Correctness Classification on LiveSQLBench SQLite
0.842AUROCFunctional Sets Confidence
Evaluation Results
| Method | Links | |
|---|---|---|
| Functional Sets ConfidenceModel=GPT-4o-mini2026.05 | 0.842 | |
| Functional EntropyModel=GPT-4o-mini2026.05 | 0.84 | |
| P(True)Model=GPT-4o2026.05 | 0.818 | |
| Functional Equivalence RateModel=GPT-4o-mini2026.05 | 0.782 | |
| Functional Sets ConfidenceModel=GPT-4o2026.05 | 0.778 | |
| Functional EntropyModel=GPT-4o2026.05 | 0.775 | |
| Functional Sets ConfidenceModel=Gemini-2.5-Pro2026.05 | 0.75 | |
| Functional Equivalence RateModel=GPT-4o2026.05 | 0.749 | |
| P(True)Model=Gemini-2.5-Flash-Lite2026.05 | 0.749 | |
| Functional EntropyModel=Gemini-2.5-Pro2026.05 | 0.745 | |
| Functional EntropyModel=Gemini-2.5-Flash-Lite2026.05 | 0.745 | |
| Functional Sets ConfidenceModel=Gemini-2.5-Flash-Lite2026.05 | 0.743 | |
| Verbalized ConfidenceModel=Gemini-2.5-Flash-Lite2026.05 | 0.739 | |
| Functional Equivalence RateModel=Gemini-2.5-Flash-Lite2026.05 | 0.738 | |
| Verbalized ConfidenceModel=GPT-4o-mini2026.05 | 0.737 | |
| Cosine SimilarityModel=Gemini-2.5-Pro2026.05 | 0.726 | |
| Cosine SimilarityModel=GPT-4o2026.05 | 0.721 | |
| Minimum Token NegentropyModel=Gemini-2.5-Flash-Lite2026.05 | 0.712 | |
| Minimum Token NegentropyModel=GPT-4o2026.05 | 0.706 | |
| Cosine SimilarityModel=GPT-4o-mini2026.05 | 0.705 | |
| Functional Equivalence RateModel=Gemini-2.5-Pro2026.05 | 0.702 | |
| P(True)Model=Gemini-2.5-Pro2026.05 | 0.695 | |
| Average Token NegentropyModel=GPT-4o-mini2026.05 | 0.694 | |
| Minimum Token NegentropyModel=GPT-4o-mini2026.05 | 0.687 | |
| P(True)Model=GPT-4o-mini2026.05 | 0.687 | |
| Cosine SimilarityModel=Gemini-2.5-Flash-Lite2026.05 | 0.683 | |
| Minimum ProbabilityModel=GPT-4o2026.05 | 0.679 | |
| Sequence ProbabilityModel=GPT-4o-mini2026.05 | 0.678 | |
| Functional Sets ConfidenceModel=Gemini-2.5-Flash2026.05 | 0.678 | |
| Functional EntropyModel=Gemini-2.5-Flash2026.05 | 0.674 | |
| Probability MarginModel=GPT-4o-mini2026.05 | 0.673 | |
| Minimum ProbabilityModel=GPT-4o-mini2026.05 | 0.672 | |
| Cosine SimilarityModel=Gemini-2.5-Flash2026.05 | 0.665 | |
| Average Token NegentropyModel=Gemini-2.5-Flash-Lite2026.05 | 0.662 | |
| Sequence ProbabilityModel=Gemini-2.5-Flash-Lite2026.05 | 0.658 | |
| Functional Equivalence RateModel=Gemini-2.5-Flash2026.05 | 0.657 | |
| Minimum Token NegentropyModel=Gemini-2.5-Flash2026.05 | 0.654 | |
| P(True)Model=Gemini-2.5-Flash2026.05 | 0.647 | |
| Sequence ProbabilityModel=GPT-4o2026.05 | 0.646 | |
| Minimum ProbabilityModel=Gemini-2.5-Flash-Lite2026.05 | 0.641 | |
| Average Token NegentropyModel=GPT-4o2026.05 | 0.632 | |
| Probability MarginModel=Gemini-2.5-Flash-Lite2026.05 | 0.629 | |
| Minimum ProbabilityModel=Gemini-2.5-Flash2026.05 | 0.628 | |
| Minimum Token NegentropyModel=Gemini-2.5-Pro2026.05 | 0.623 | |
| Probability MarginModel=GPT-4o2026.05 | 0.616 | |
| Minimum ProbabilityModel=Gemini-2.5-Pro2026.05 | 0.602 | |
| Verbalized ConfidenceModel=Gemini-2.5-Pro2026.05 | 0.596 | |
| Verbalized ConfidenceModel=GPT-4o2026.05 | 0.596 | |
| Average Token NegentropyModel=Gemini-2.5-Flash2026.05 | 0.558 | |
| Sequence ProbabilityModel=Gemini-2.5-Flash2026.05 | 0.554 | |
| Probability MarginModel=Gemini-2.5-Flash2026.05 | 0.546 | |
| Verbalized ConfidenceModel=Gemini-2.5-Flash2026.05 | 0.535 | |
| Sequence ProbabilityModel=Gemini-2.5-Pro2026.05 | 0.529 | |
| Average Token NegentropyModel=Gemini-2.5-Pro2026.05 | 0.528 | |
| Probability MarginModel=Gemini-2.5-Pro2026.05 | 0.517 |