Holistic Technical Quality Evaluation on ICLR 2025
3.35OriginalityPaperOrchestra (PlotOff)
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| PaperOrchestra (PlotOff)Reviewer Framework=ScholarPeer Reviewer, Setting=Sparse Idea Setting2026.04 | 3.35 | 2.96 | 3.65 | 3.59 | 2.94 | 3.48 | 3.37 | 7.03 | 81 | |
| Original (GT)Reviewer Framework=ScholarPeer Reviewer, Setting=Sparse Idea Setting2026.04 | 3.33 | 3.14 | 3.76 | 3.74 | 3.05 | 3.59 | 3.57 | 7.48 | 94 | |
| SingleAgentReviewer Framework=ScholarPeer Reviewer, Setting=Sparse Idea Setting2026.04 | 3.26 | 2.73 | 3.27 | 3.48 | 2.71 | 3.2 | 3.25 | 6.35 | 72 | |
| AI Scientist-v2Reviewer Framework=ScholarPeer Reviewer, Setting=Sparse Idea Setting2026.04 | 3.23 | 2.62 | 3.28 | 3.41 | 2.6 | 3.23 | 3.11 | 6.04 | 64 | |
| Original (GT)Reviewer Framework=AI Scientist-v2 Reviewer, Setting=Sparse Idea Setting2026.04 | 2.99 | 2.79 | 3.41 | 3.03 | 2.76 | 3.3 | 2.94 | 5.81 | 63 | |
| SingleAgentReviewer Framework=AI Scientist-v2 Reviewer, Setting=Sparse Idea Setting2026.04 | 2.7 | 1.67 | 2.2 | 2.44 | 1.76 | 2.19 | 2.09 | 3.22 | 4 | |
| PaperOrchestra (PlotOff)Reviewer Framework=AI Scientist-v2 Reviewer, Setting=Sparse Idea Setting2026.04 | 2.68 | 2.1 | 2.82 | 2.57 | 2.12 | 2.79 | 2.37 | 4.1 | 22 | |
| AI Scientist-v2Reviewer Framework=AI Scientist-v2 Reviewer, Setting=Sparse Idea Setting2026.04 | 2.58 | 1.76 | 2.27 | 2.46 | 1.82 | 2.29 | 2.16 | 3.42 | 11 |