Logic on Synlogic
67.4AccuracyLowest Centroid
Evaluation results
| Method | Links | |
|---|---|---|
| Lowest CentroidModel=GPT-OSS-120B, Number of Samples=642026.04 | 67.4 | |
| Tail ConfidenceModel=GPT-OSS-120B, Number of Samples=642026.04 | 66 | |
| Bottom WindowModel=GPT-OSS-120B, Number of Samples=642026.04 | 64.3 | |
| Pass@1Model=GPT-OSS-120B, Number of Samples=642026.04 | 63.1 | |
| Greedy DecodingModel=GPT-OSS-120B, Number of Samples=642026.04 | 62 | |
| Self-CertaintyModel=GPT-OSS-120B, Number of Samples=642026.04 | 61.4 | |
| Lowest CentroidModel=QWQ-32B, Number of Samples=642026.04 | 56 | |
| Bottom WindowModel=QWQ-32B, Number of Samples=642026.04 | 55.7 | |
| Self-CertaintyModel=Qwen3-14B, Number of Samples=642026.04 | 50.3 | |
| Tail ConfidenceModel=Qwen3-14B, Number of Samples=642026.04 | 50.3 | |
| Lowest CentroidModel=Olmo-3.1-Think, Number of Samples=642026.04 | 50 | |
| Bottom WindowModel=Olmo-3.1-Think, Number of Samples=642026.04 | 48.9 | |
| Bottom WindowModel=Qwen3-14B, Number of Samples=642026.04 | 48.9 | |
| Self-CertaintyModel=QWQ-32B, Number of Samples=642026.04 | 48.6 | |
| Tail ConfidenceModel=Olmo-3.1-Think, Number of Samples=642026.04 | 46 | |
| Pass@1Model=QWQ-32B, Number of Samples=642026.04 | 46 | |
| Lowest CentroidModel=Qwen3-14B, Number of Samples=642026.04 | 45.4 | |
| Tail ConfidenceModel=QWQ-32B, Number of Samples=642026.04 | 44.6 | |
| Pass@1Model=Qwen3-14B, Number of Samples=642026.04 | 44.4 | |
| Self-CertaintyModel=Olmo-3.1-Think, Number of Samples=642026.04 | 42.3 | |
| Pass@1Model=Olmo-3.1-Think, Number of Samples=642026.04 | 42.1 | |
| Greedy DecodingModel=QWQ-32B, Number of Samples=642026.04 | 40.9 | |
| Greedy DecodingModel=Olmo-3.1-Think, Number of Samples=642026.04 | 40.3 | |
| Greedy DecodingModel=Qwen3-14B, Number of Samples=642026.04 | 38.9 | |
| Tail ConfidenceModel=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 38.9 | |
| Bottom WindowModel=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 36.6 | |
| Lowest CentroidModel=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 33.4 | |
| Pass@1Model=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 23.1 | |
| Self-CertaintyModel=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 20 | |
| Greedy DecodingModel=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 16.3 |