Literature Review Quality Assessment on PaperWritingBench
75.36Citation Practices ScorePaperOrchestra (PlotOff)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| PaperOrchestra (PlotOff)Judge Model=Gemini-3.1-Pro, Setting=Sparse Idea Setting2026.04 | 75.36 | 70.15 | 79.01 | 83.74 | 82.68 | 81.15 | 78.3 | |
| Human (GT)Judge Model=Gemini-3.1-Pro, Setting=Sparse Idea Setting2026.04 | 71.66 | 68.3 | 74.6 | 80.54 | 82.22 | 80.67 | 75.18 | |
| PaperOrchestra (PlotOff)Judge Model=Gemini-3.1-Pro, Setting=Sparse Idea Setting2026.04 | 71.15 | 65.71 | 77.34 | 82.02 | 81.94 | 80.08 | 76.23 | |
| Human (GT)Judge Model=Gemini-3.1-Pro, Setting=Sparse Idea Setting2026.04 | 69.88 | 71.55 | 65.8 | 74.31 | 75.55 | 78.25 | 71.08 | |
| PaperOrchestra (PlotOff)Judge Model=GPT5, Setting=Sparse Idea Setting2026.04 | 63.88 | 64.13 | 60.04 | 77.46 | 59.18 | 72.65 | 53.99 | |
| Human (GT)Judge Model=GPT5, Setting=Sparse Idea Setting2026.04 | 63.47 | 63.86 | 57.36 | 76.16 | 61.34 | 73.23 | 53.66 | |
| PaperOrchestra (PlotOff)Judge Model=GPT5, Setting=Sparse Idea Setting2026.04 | 62.54 | 62.12 | 60.58 | 78.7 | 60.21 | 73.58 | 54.15 | |
| Human (GT)Judge Model=GPT5, Setting=Sparse Idea Setting2026.04 | 61.61 | 60.91 | 58.75 | 77.19 | 62.91 | 73.11 | 55.53 | |
| SingleAgentJudge Model=GPT5, Setting=Sparse Idea Setting2026.04 | 49.69 | 47.01 | 54.49 | 76.55 | 56.62 | 70.6 | 44.33 | |
| AI Scientist-v2Judge Model=GPT5, Setting=Sparse Idea Setting2026.04 | 49.55 | 48.26 | 54.18 | 76.62 | 56.08 | 69.86 | 44.29 | |
| SingleAgentJudge Model=GPT5, Setting=Sparse Idea Setting2026.04 | 49.34 | 45.97 | 54.1 | 77.79 | 56.47 | 71.52 | 44.3 | |
| AI Scientist-v2Judge Model=GPT5, Setting=Sparse Idea Setting2026.04 | 49.18 | 47.59 | 53.71 | 75.71 | 55.8 | 69 | 43.57 | |
| AI Scientist-v2Judge Model=Gemini-3.1-Pro, Setting=Sparse Idea Setting2026.04 | 39.24 | 34.25 | 54.83 | 77.21 | 65.16 | 69.12 | 45.43 | |
| AI Scientist-v2Judge Model=Gemini-3.1-Pro, Setting=Sparse Idea Setting2026.04 | 37.6 | 32.27 | 51.84 | 75.74 | 62.96 | 67.2 | 42.98 | |
| SingleAgentJudge Model=Gemini-3.1-Pro, Setting=Sparse Idea Setting2026.04 | 36.83 | 30.1 | 52.93 | 77.03 | 66.45 | 71.4 | 43.04 | |
| SingleAgentJudge Model=Gemini-3.1-Pro, Setting=Sparse Idea Setting2026.04 | 33.03 | 26.35 | 49.8 | 75.38 | 61.14 | 69.7 | 38.91 |