Open-ended generation on Zebra-Logic
0.75FDRSGR
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| SGRModel=Qwen3-4B-Instruct-2507, Uncertainty Score=logits-based2025.10 | 0.75 | 6.52 | 5.46 | — | |
| SGRModel=Qwen3-4B-Thinking-2507, Uncertainty Score=logits-based2025.10 | 2.45 | 32.11 | 30.02 | — | |
| FDR searchModel=Qwen3-4B-Thinking-2507, Uncertainty Score=logits-based2025.10 | 3.05 | 50.24 | 46.28 | — | |
| FDR searchModel=Qwen3-4B-Instruct-2507, Uncertainty Score=logits-based2025.10 | 4.23 | 42.36 | 35.18 | — | |
| Conformal LabelingModel=Qwen3-4B-Instruct-2507, Uncertainty Score=logits-based2025.10 | 4.76 | 47.8 | 39.94 | — | |
| Conformal LabelingModel=DeepSeek-R1-Distill-Qwen-32B, Uncertainty Score=logits-based2025.10 | 4.78 | 0.01 | 0.02 | — | |
| Conformal LabelingModel=Qwen3-4B-Thinking-2507, Uncertainty Score=logits-based2025.10 | 4.95 | 67.83 | 63.73 | — | |
| FDR searchModel=DeepSeek-R1-Distill-Qwen-32B, Uncertainty Score=logits-based2025.10 | 67.7 | 0.05 | 0.32 | — | |
| SGRModel=DeepSeek-R1-Distill-Qwen-32B, Uncertainty Score=logits-based2025.10 | 69.33 | 0.03 | 0.03 | — | |
| AI onlyModel=Qwen3-4B-Instruct-25072025.10 | — | — | — | 20.9 | |
| AI onlyModel=Qwen3-4B-Thinking-25072025.10 | — | — | — | 11 | |
| AI onlyModel=DeepSeek-R1-Distill-Qwen-32B2025.10 | — | — | — | 31.4 |