Automated Peer Review Evaluation on DeepReview-13K 1.0 (test)
8.33H-Max Technical AccuracyScholarPeer
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| ScholarPeerCategory=Ours, LLM Judge=Gemini 3 Pro2026.01 | 8.33 | 8.61 | 8.21 | 8.64 | 8.51 | 0.42 | 0.29 | |
| Stanford Agent ReviewerCategory=Multi Agent, LLM Judge=Gemini 3 Pro2026.01 | 8.12 | 8.26 | 8.04 | 8.1 | 8.14 | 0.4 | — | |
| Gemini 3 ProCategory=Single Agent, LLM Judge=Gemini 3 Pro2026.01 | 7.59 | 6.79 | 6.89 | 6.3 | 6.88 | 0.38 | 0.24 | |
| AI Scientist v2 (Gemini 3 Flash)Category=Multi Agent, LLM Judge=Gemini 3 Pro2026.01 | 7.59 | 6.95 | 7.06 | 6.51 | 7.01 | 0.33 | 0.22 | |
| AI Scientist v2 (Gemini 3 Pro)Category=Multi Agent, LLM Judge=Gemini 3 Pro2026.01 | 7.56 | 6.85 | 6.68 | 6.32 | 6.78 | 0.33 | 0.22 | |
| Agent Review (Gemini 3 Pro)Category=Multi Agent, LLM Judge=Gemini 3 Pro2026.01 | 7.55 | 6.81 | 6.72 | 6.28 | 6.77 | 0.39 | 0.22 | |
| Gemini 3 FlashCategory=Single Agent, LLM Judge=Gemini 3 Pro2026.01 | 7.24 | 6.35 | 6.51 | 6.07 | 6.46 | 0.35 | 0.24 | |
| Agent Review (Claude 4.5 Sonnet)Category=Multi Agent, LLM Judge=Gemini 3 Pro2026.01 | 7.21 | 6.51 | 6.75 | 6.55 | 6.64 | 0.37 | 0.21 | |
| Claude 4.5 SonnetCategory=Single Agent, LLM Judge=Gemini 3 Pro2026.01 | 7.09 | 6.46 | 6.87 | 6.4 | 6.63 | 0.29 | 0.22 | |
| Agent Review (Gemini 3 Flash)Category=Multi Agent, LLM Judge=Gemini 3 Pro2026.01 | 7.01 | 6.26 | 6.25 | 6.14 | 6.31 | 0.39 | 0.23 | |
| AI Scientist v2 (Claude 4.5 Sonnet)Category=Multi Agent, LLM Judge=Gemini 3 Pro2026.01 | 7 | 6.65 | 6.96 | 6.39 | 6.69 | 0.37 | 0.21 | |
| ScholarPeerCategory=Ours2026.01 | 5.79 | 5.88 | 5.86 | 6.49 | 6.14 | 0.42 | 0.29 | |
| Stanford Agent ReviewerCategory=Multi Agent2026.01 | 5.5 | 5.39 | 6.22 | 5.56 | 5.89 | 0.4 | — | |
| DeepReviewer-14BCategory=Fine-tuned, LLM Judge=Gemini 3 Pro2026.01 | 5.19 | 4.75 | 4.47 | 4.71 | 4.36 | 0.36 | 0.02 | |
| AI Scientist v2Category=Multi Agent, Backbone=Gemini 3 Flash2026.01 | 5 | 4.63 | 4.67 | 4.38 | 4.69 | 0.33 | 0.22 | |
| Gemini 3 ProCategory=Single Agent2026.01 | 4.83 | 4.33 | 4.52 | 3.9 | 4.46 | 0.38 | 0.24 | |
| Agent ReviewCategory=Multi Agent, Backbone=Gemini 3 Pro2026.01 | 4.74 | 4.3 | 4.4 | 4.04 | 4.41 | 0.39 | 0.22 | |
| AI Scientist v2Category=Multi Agent, Backbone=Claude Sonnet 4.52026.01 | 4.71 | 4.49 | 4.68 | 4.6 | 4.62 | 0.37 | 0.21 | |
| Claude Sonnet 4.5Category=Single Agent2026.01 | 4.67 | 4.38 | 4.6 | 4.37 | 4.5 | 0.29 | 0.22 | |
| Gemini 3 FlashCategory=Single Agent2026.01 | 4.65 | 4.16 | 4.41 | 3.82 | 4.35 | 0.35 | 0.24 | |
| AI Scientist v2Category=Multi Agent, Backbone=Gemini 3 Pro2026.01 | 4.59 | 4.25 | 4.34 | 3.98 | 4.35 | 0.33 | 0.22 | |
| Agent ReviewCategory=Multi Agent, Backbone=Claude Sonnet 4.52026.01 | 4.57 | 4.49 | 4.69 | 4.38 | 4.56 | 0.37 | 0.21 | |
| Agent ReviewCategory=Multi Agent, Backbone=Gemini 3 Flash2026.01 | 4.38 | 4.18 | 4.27 | 3.93 | 4.23 | 0.39 | 0.23 | |
| DeepReviewer-7BCategory=Fine-tuned, LLM Judge=Gemini 3 Pro2026.01 | 3.97 | 3.81 | 3.58 | 4.13 | 3.41 | 0.31 | 0.02 | |
| DeepReviewer-14BCategory=Fine-tuned2026.01 | 3.8 | 3.88 | 3.82 | 3.4 | 3.69 | 0.36 | 0.02 | |
| DeepReviewer-7BCategory=Fine-tuned2026.01 | 3.31 | 3.51 | 3.5 | 3.34 | 3.41 | 0.31 | 0.02 | |
| CycleReviewer-70BCategory=Fine-tuned, LLM Judge=Gemini 3 Pro2026.01 | 2.27 | 2.38 | 2.27 | 3.45 | 2.17 | 0.27 | 0.01 | |
| CycleReviewer-8BCategory=Fine-tuned, LLM Judge=Gemini 3 Pro2026.01 | 2.13 | 2.29 | 2.21 | 3.42 | 2.08 | 0.27 | 0.01 | |
| CycleReviewer-70BCategory=Fine-tuned2026.01 | 2.11 | 2.19 | 1.92 | 2.34 | 1.91 | 0.27 | 0.01 | |
| CycleReviewer-8BCategory=Fine-tuned2026.01 | 1.96 | 2.08 | 1.9 | 2.28 | 1.81 | 0.27 | 0.01 |