Research Idea Evaluation on ScholarIdeas-AI contribution rubrics
3.45Coverage (mean)ScholarEval-GPT-4.1
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ScholarEval-GPT-4.1Backbone=GPT-4.12025.10 | 3.45 | 1.34 | |
| ScholarEval-Claude-4-SonnetBackbone=Claude-4-Sonnet2025.10 | 3.34 | 1.34 | |
| ScholarEval ClaudeMethod Category=ScholarEval, Backbone=Claude-4-Sonnet2025.10 | 2.91 | — | |
| ScholarEval GPT-4.1Method Category=ScholarEval, Backbone=GPT-4.12025.10 | 2.84 | — | |
| GPT-4.12025.10 | 2.78 | 1.15 | |
| ScholarEval ClaudeMethod Category=ScholarEval, Backbone=Claude-4-Sonnet2025.10 | 2.77 | — | |
| ScholarEval GPT-4.1Method Category=ScholarEval, Backbone=GPT-4.12025.10 | 2.72 | — | |
| OpenAI Deep Research2025.10 | 2.66 | 1.06 | |
| Claude-4-Sonnet2025.10 | 2.57 | 1.05 | |
| ScholarEval-Llama-3.3Backbone=Llama-3.32025.10 | 2.52 | 1.17 | |
| GPT-4o-search-preview2025.10 | 2.36 | 1.05 | |
| OpenAI Deep ResearchMethod Category=Deep Research, Backbone=OpenAI Deep Research2025.10 | 2.35 | — | |
| DR TuluMethod Category=Deep Research, Backbone=DR Tulu2025.10 | 2.35 | — | |
| DR TuluMethod Category=Deep Research, Backbone=DR Tulu2025.10 | 2.33 | — | |
| ScholarEval GPT-5.1Method Category=ScholarEval, Backbone=GPT-5.1 Instant2025.10 | 2.31 | — | |
| OpenAI Deep ResearchMethod Category=Deep Research, Backbone=OpenAI Deep Research2025.10 | 2.28 | — | |
| Claude-4-SonnetMethod Category=Language Models, Backbone=Claude-4-Sonnet2025.10 | 2.28 | — | |
| Idea Novelty Checker2025.10 | 2.27 | 1.15 | |
| Llama-3.3-70B2025.10 | 2.22 | 1.04 | |
| GPT-4.1Method Category=Language Models, Backbone=GPT-4.12025.10 | 2.21 | — | |
| GPT-4.1Method Category=Language Models, Backbone=GPT-4.12025.10 | 2.18 | — | |
| Claude-4-SonnetMethod Category=Language Models, Backbone=Claude-4-Sonnet2025.10 | 2.18 | — | |
| ScholarEval GPT-5.1Method Category=ScholarEval, Backbone=GPT-5.1 Instant2025.10 | 2.17 | — | |
| GPT-5.1 InstantMethod Category=Language Models, Backbone=GPT-5.1 Instant2025.10 | 2.09 | — | |
| ScholarEval LlamaMethod Category=ScholarEval, Backbone=Llama-3.3-70B2025.10 | 2.06 | — | |
| ScholarEval LlamaMethod Category=ScholarEval, Backbone=Llama-3.3-70B2025.10 | 2.04 | — | |
| GPT-4o-search-previewMethod Category=Web-connected LM, Backbone=GPT-4o-search-preview2025.10 | 1.95 | — | |
| GPT-5.1 InstantMethod Category=Language Models, Backbone=GPT-5.1 Instant2025.10 | 1.94 | — | |
| GPT-4o-search-previewMethod Category=Web-connected LM, Backbone=GPT-4o-search-preview2025.10 | 1.9 | — | |
| Llama-3.3-70BMethod Category=Language Models, Backbone=Llama-3.3-70B2025.10 | 1.88 | — | |
| Llama-3.3-70BMethod Category=Language Models, Backbone=Llama-3.3-70B2025.10 | 1.83 | — |