Scientific question generation on ICLR papers 2024 (test)
0.54EffectivenessHuman questions
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Human questionsReasoning Mode=Not Applicable2026.01 | 0.54 | 0.46 | 0.57 | 1.57 | 28.21 | |
| o3Reasoning Mode=Medium2026.01 | 0.32 | 0.12 | 0.36 | 0.8 | 16.81 | |
| IntelliAsk-32BReasoning Mode=Default2026.01 | 0.27 | 0.13 | 0.26 | 0.66 | 21.37 | |
| Gemini 2.5 ProReasoning Mode=Default2026.01 | 0.26 | 0.13 | 0.21 | 0.6 | 25.75 | |
| Qwen2.5-32BReasoning Mode=No reasoning2026.01 | 0.02 | 0.01 | 0.02 | 0.05 | 54.96 |