Idea Generation on 30 AI research queries (test)
100Win RateSCION
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SCIONbaseline=AutoResearchClaw, llm_judge=GPT-5.5, evaluation_protocol=pairwise novelty evaluation, positional_bias_reduction=swapped answer order2026.07 | 100 | 0 | 0 | |
| SCIONbaseline=ScienceClaw, llm_judge=GPT-5.5, evaluation_protocol=pairwise novelty evaluation, positional_bias_reduction=swapped answer order2026.07 | 100 | 0 | 0 | |
| SCIONbaseline=EvoScientist, llm_judge=GPT-5.5, evaluation_protocol=pairwise novelty evaluation, positional_bias_reduction=swapped answer order2026.07 | 98.33 | 0 | 1.67 | |
| SCIONbaseline=ARIS, llm_judge=GPT-5.5, evaluation_protocol=pairwise novelty evaluation, positional_bias_reduction=swapped answer order2026.07 | 91.67 | 1.67 | 6.67 | |
| SCIONbaseline=AI-Researcher, llm_judge=GPT-5.5, evaluation_protocol=pairwise novelty evaluation, positional_bias_reduction=swapped answer order2026.07 | 88.33 | 0 | 11.67 | |
| SCIONbaseline=InternAgent-1.0, llm_judge=GPT-5.5, evaluation_protocol=pairwise novelty evaluation, positional_bias_reduction=swapped answer order2026.07 | 88.33 | 0 | 11.67 | |
| SCIONbaseline=AI-Scientist-v2, llm_judge=GPT-5.5, evaluation_protocol=pairwise novelty evaluation, positional_bias_reduction=swapped answer order2026.07 | 86.67 | 0 | 13.33 | |
| SCIONbaseline=DR-Claw, llm_judge=GPT-5.5, evaluation_protocol=pairwise novelty evaluation, positional_bias_reduction=swapped answer order2026.07 | 61.67 | 1.67 | 36.67 |