Automated Peer Review on DeepReview-13K 2025 (test)
98.7Technical Accuracy WinCycleReviewer-8B
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CycleReviewer-8BCategory=Fine-tuned, Size=8B2026.01 | 98.7 | 0.9 | 100 | 0 | 100 | 0 | 98.5 | 0.3 | 99.7 | 0.2 | |
| CycleReviewer-70BCategory=Fine-tuned, Size=70B2026.01 | 98.7 | 1 | 99.8 | 0.2 | 99.8 | 0 | 98.6 | 0.4 | 99.5 | 0.4 | |
| DeepReviewer-7BCategory=Fine-tuned, Size=7B2026.01 | 93.2 | 4.2 | 98.1 | 1.7 | 97.6 | 1.3 | 96 | 1.7 | 97.6 | 2.2 | |
| DeepReviewer-14BCategory=Fine-tuned, Size=14B2026.01 | 77.9 | 11.7 | 89.2 | 9.3 | 85.9 | 10.9 | 90.3 | 4.8 | 91.5 | 8.3 | |
| AI Scientist v2 (Gemini 3 Pro)Category=Multi Agent, Backbone=Gemini 3 Pro2026.01 | 49.2 | 25.8 | 83.8 | 13.2 | 82.3 | 11.4 | 78 | 10.7 | 81.6 | 18 | |
| Agent Review (Gemini 3 Pro)Category=Multi Agent, Backbone=Gemini 3 Pro2026.01 | 45.3 | 28.2 | 83.6 | 13 | 80.1 | 13.4 | 79 | 10.1 | 80 | 18.9 | |
| Claude Sonnet 4.5Category=Single Agent2026.01 | 44.7 | 24.4 | 74.1 | 20.3 | 53.8 | 36.5 | 78.2 | 12.7 | 75.1 | 23.4 | |
| Gemini 3 ProCategory=Single Agent2026.01 | 40.2 | 33 | 76.9 | 18.1 | 69.1 | 22.5 | 74.9 | 11.8 | 73 | 26.1 | |
| Gemini 3 FlashCategory=Single Agent2026.01 | 39.5 | 32.4 | 81.6 | 16.7 | 74.9 | 18.7 | 77.9 | 13 | 78.3 | 21.7 | |
| AI Scientist v2 (Gemini 3 Flash)Category=Multi Agent, Backbone=Gemini 3 Flash2026.01 | 38.1 | 34.2 | 76 | 20.4 | 63.7 | 25.8 | 71.2 | 15.5 | 71 | 27 | |
| AI Scientist v2 (Claude Sonnet 4.5)Category=Multi Agent, Backbone=Claude Sonnet 4.52026.01 | 37.2 | 26.1 | 69.8 | 26.6 | 44.7 | 46.7 | 72.9 | 17.1 | 65.8 | 34.2 | |
| Stanford Agent ReviewerCategory=Multi Agent, Access=Browser based, Sample Size=50 papers sampled from DeepReview-13K2026.01 | 36 | 32 | 52 | 46 | 38 | 46 | 60 | 34 | 54 | 46 | |
| Agent Review (Gemini 3 Flash)Category=Multi Agent, Backbone=Gemini 3 Flash2026.01 | 34.7 | 27.8 | 77.1 | 20.3 | 67.6 | 26.9 | 75.4 | 13.4 | 76.5 | 23 | |
| Agent Review (Claude Sonnet 4.5)Category=Multi Agent, Backbone=Claude Sonnet 4.52026.01 | 33.5 | 33 | 59.4 | 38.6 | 32.5 | 55.8 | 69.1 | 16.2 | 57.9 | 41.6 |