Scientific question generation on IntelliReward (test)
0.28Efforto3
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| o3Reasoning=Medium2026.01 | 0.28 | 0.14 | 0.3 | 0.72 | 16.81 | |
| IntelliAsk-32BReasoning=Default2026.01 | 0.23 | 0.12 | 0.2 | 0.55 | 21.37 | |
| gemini-2.5-proReasoning=Default2026.01 | 0.22 | 0.11 | 0.18 | 0.51 | 25.75 | |
| deepseek-chat-v3.1Reasoning=Default2026.01 | 0.11 | 0.2 | 0.17 | 0.48 | 36.83 | |
| gpt-5Reasoning=Default2026.01 | 0.09 | 0.2 | 0.16 | 0.45 | 18.63 | |
| claude-3.7-sonnetReasoning=No2026.01 | 0.09 | 0.18 | 0.15 | 0.42 | 45.14 | |
| llama-4-maverickReasoning=No2026.01 | 0.09 | 0.17 | 0.15 | 0.41 | 48.48 | |
| gpt-oss-120bReasoning=Medium2026.01 | 0.08 | 0.15 | 0.12 | 0.35 | 22.99 | |
| claude-3.7-sonnetReasoning=Default2026.01 | 0.08 | 0.16 | 0.13 | 0.37 | 47.13 | |
| gemini-2.5-flashReasoning=No2026.01 | 0.08 | 0.15 | 0.15 | 0.38 | 39.06 | |
| gpt-4.1Reasoning=No2026.01 | 0.07 | 0.12 | 0.12 | 0.31 | 31.73 | |
| grok-4Reasoning=No2026.01 | 0.07 | 0.14 | 0.12 | 0.33 | 35.47 | |
| gpt-oss-20bReasoning=Medium2026.01 | 0.06 | 0.11 | 0.1 | 0.27 | 24.81 | |
| Qwen3-32BReasoning=Default2026.01 | 0.05 | 0.13 | 0.09 | 0.28 | 26.73 | |
| IntelliAsk-7BReasoning=No2026.01 | 0.03 | 0.07 | 0.07 | 0.17 | 27.44 | |
| OpenReviewer-8BReasoning=No2026.01 | 0 | 0 | 0.1 | 0.1 | 51.14 | |
| DeepReviewer-7BReasoning=No2026.01 | 0 | 0 | 0.1 | 0.1 | 48.14 | |
| Qwen2.5-7BReasoning=No2026.01 | 0 | 0.01 | 0.01 | 0.02 | 49.93 | |
| Qwen2.5-7B SFTReasoning=No2026.01 | 0 | 0.01 | 0.02 | 0.03 | 42.11 |