Code on LiveCode
83.6AccuracyLowest Centroid
Evaluation results
| Method | Links | |
|---|---|---|
| Lowest CentroidModel=GPT-OSS-120B, Number of Samples=322026.04 | 83.6 | |
| Greedy DecodingModel=GPT-OSS-120B, Number of Samples=322026.04 | 82 | |
| Bottom WindowModel=GPT-OSS-120B, Number of Samples=322026.04 | 81.3 | |
| Pass@1Model=GPT-OSS-120B, Number of Samples=322026.04 | 81.2 | |
| Self-CertaintyModel=GPT-OSS-120B, Number of Samples=322026.04 | 80.3 | |
| Tail ConfidenceModel=GPT-OSS-120B, Number of Samples=322026.04 | 80.1 | |
| Self-CertaintyModel=Qwen3-14B, Number of Samples=322026.04 | 79.5 | |
| Tail ConfidenceModel=Qwen3-14B, Number of Samples=322026.04 | 78.6 | |
| Lowest CentroidModel=Qwen3-14B, Number of Samples=322026.04 | 78.1 | |
| Self-CertaintyModel=QWQ-32B, Number of Samples=322026.04 | 78.1 | |
| Lowest CentroidModel=QWQ-32B, Number of Samples=322026.04 | 78 | |
| Bottom WindowModel=QWQ-32B, Number of Samples=322026.04 | 77.9 | |
| Bottom WindowModel=Qwen3-14B, Number of Samples=322026.04 | 77.7 | |
| Tail ConfidenceModel=QWQ-32B, Number of Samples=322026.04 | 77.4 | |
| Lowest CentroidModel=Olmo-3.1-Think, Number of Samples=322026.04 | 77.2 | |
| Pass@1Model=Qwen3-14B, Number of Samples=322026.04 | 77.2 | |
| Pass@1Model=QWQ-32B, Number of Samples=322026.04 | 76.7 | |
| Self-CertaintyModel=Olmo-3.1-Think, Number of Samples=322026.04 | 76.6 | |
| Tail ConfidenceModel=Olmo-3.1-Think, Number of Samples=322026.04 | 76.3 | |
| Bottom WindowModel=Olmo-3.1-Think, Number of Samples=322026.04 | 75.9 | |
| Pass@1Model=Olmo-3.1-Think, Number of Samples=322026.04 | 74.2 | |
| Greedy DecodingModel=QWQ-32B, Number of Samples=322026.04 | 73.2 | |
| Greedy DecodingModel=Qwen3-14B, Number of Samples=322026.04 | 72.6 | |
| Greedy DecodingModel=Olmo-3.1-Think, Number of Samples=322026.04 | 71.6 | |
| Tail ConfidenceModel=Ministral-3-14B-Instruct, Number of Samples=322026.04 | 46.8 | |
| Bottom WindowModel=Ministral-3-14B-Instruct, Number of Samples=322026.04 | 44.5 | |
| Lowest CentroidModel=Ministral-3-14B-Instruct, Number of Samples=322026.04 | 44.4 | |
| Pass@1Model=Ministral-3-14B-Instruct, Number of Samples=322026.04 | 40.3 | |
| Self-CertaintyModel=Ministral-3-14B-Instruct, Number of Samples=322026.04 | 40 | |
| Greedy DecodingModel=Ministral-3-14B-Instruct, Number of Samples=322026.04 | 39.7 |