Scientific Idea Generation on Neurips 2025
4.28Absolute Novelty ScoreOur Method - BoN(10)
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Our Method - BoN(10)Backbone=Qwen 2.5 14B, Quantization=4-bit, Best-of-N=102026.04 | 4.28 | 3.88 | 4.42 | 5 | 1 | 5 | — | — | — | — | |
| LDC2026.04 | 4.25 | 3.9 | 4.53 | 3.29 | 1.94 | 1.55 | — | — | — | — | |
| Our MethodModel=unsloth-Qwen2.5-7B, Strategy=BoN (10)2026.04 | 4.15 | 3.9 | 4.59 | 5 | 1.05 | 5 | — | — | — | — | |
| GPT Researcher2026.04 | 4.12 | 3.92 | 4.53 | 4.77 | 1.27 | 3.96 | — | — | — | — | |
| SFTBackbone=Qwen 2.5 14B, Mode=Supervised fine-tuning2026.04 | 4.08 | 3.82 | 4.28 | 2.11 | 3.18 | 1.85 | — | — | — | — | |
| Base ModelModel=unsloth-Qwen2.5-7B2026.04 | 4.08 | 4.03 | 4.44 | 1.93 | 5 | 2.83 | — | — | — | — | |
| AI Scientist V22026.04 | 4.05 | 4.18 | 4.38 | 2.89 | 3.54 | 3.27 | — | — | — | — | |
| GPT Researcher2026.04 | 4.03 | 3.79 | 4.54 | 3.87 | 1 | 3.04 | — | — | — | — | |
| Research Agent2026.04 | 3.92 | 3.75 | 4.3 | 2.44 | 3.01 | 1 | — | — | — | — | |
| AI Scientist V22026.04 | 3.92 | 4.2 | 4.28 | 1.08 | 5 | 1.33 | — | — | — | — | |
| Base Model 14B 4-bitBackbone=Qwen 2.5 14B, Quantization=4-bit2026.04 | 3.88 | 4.12 | 4.35 | 1 | 4.2 | 1.44 | — | — | — | — | |
| Research Agent2026.04 | 3.64 | 3.51 | 3.77 | 1 | 1.97 | 1 | — | — | — | — | |
| Agentic-IdeationBackbone=Qwen3-8B2026.06 | — | — | — | — | — | — | 6.56 | 7.88 | 6.21 | 6.64 | |
| Qwen-3-8BType=Backbone model2026.06 | — | — | — | — | — | — | 5.62 | 7.08 | 5.59 | 5.9 | |
| ResearchAgentUnderlying LLM=GPT-4o2026.06 | — | — | — | — | — | — | 5.67 | 7.13 | 5.64 | 6.01 | |
| SciMONUnderlying LLM=GPT-4o2026.06 | — | — | — | — | — | — | 5.72 | 7.05 | 5.54 | 5.87 | |
| SciPIPUnderlying LLM=GPT-4o2026.06 | — | — | — | — | — | — | 5.82 | 6.92 | 5.71 | 5.97 | |
| VirSciUnderlying LLM=GPT-4o2026.06 | — | — | — | — | — | — | 5.71 | 6.89 | 5.67 | 5.93 |