Scientific Ideation on Scientific Ideation 60 Samples Human-Validated (test)
3.63NoveltyGround Truth
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Ground Truth2026.03 | 3.63 | 3.69 | 3.44 | 3.58 | — | |
| Claude-3.5-SonnetBackbone=claude-3-5-sonnet-2024-10-222026.03 | 3.39 | 3.07 | 2.82 | 3.09 | 3.2 | |
| MoRIws=0.7, we=0.3, entropy mask=Top-25%, training steps=4002026.03 | 3.31 | 3.16 | 3.11 | 3.19 | — | |
| Full-SFT2026.03 | 3.1 | 2.87 | 3.02 | 2.99 | 6.7 | |
| AI-Scientist-V2(Claude)Backbone=claude-3-5-haiku-2024-10-222026.03 | 2.74 | 2.46 | 2.89 | 2.7 | 18.1 | |
| ResearchAgentBackbone=gpt-4o-2024-08-062026.03 | 2.66 | 2.51 | 2.63 | 2.6 | 22.7 | |
| GPT-4oBackbone=gpt-4o-2024-08-062026.03 | 2.51 | 2.78 | 2.79 | 2.69 | 18.6 | |
| AI-Scientist-V2(GPT)Backbone=gpt-4o-2024-08-062026.03 | 2.48 | 2.67 | 2.98 | 2.71 | 17.7 | |
| VirSciBackbone=gpt-4o-2024-08-062026.03 | 2.21 | 2.26 | 2.28 | 2.25 | 41.8 |