Paper Quality Evaluation on ICLR 2025 (test)
48.08Kendall Tau CorrelationGraphReview
Evaluation Results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GraphReviewIntrinsic Quality=true, Synchronic Links=true, Diachronic Links=true, Review Text=true, Data Leakage Risk=low risk2026.05 | 48.08 | — | — | — | 85.47 | — | — | — | — | 66.26 | — | — | — | 78.93 | |
| GPT-5-MiniIntrinsic Quality=true, Synchronic Links=true, Diachronic Links=false, Review Text=false, Data Leakage Risk=high risk2026.05 | 33.17 | — | — | — | 69.82 | — | — | — | — | 41.36 | — | — | — | 58.74 | |
| DeepReview-14BIntrinsic Quality=true, Synchronic Links=developed but not enabled, Diachronic Links=true, Review Text=false, Data Leakage Risk=low risk2026.05 | 29.91 | — | — | — | 66.57 | — | — | — | — | 39.95 | — | — | — | 55.4 | |
| NAIPv2-8BIntrinsic Quality=true, Synchronic Links=false, Diachronic Links=false, Review Text=false, Data Leakage Risk=low risk2026.05 | 29.28 | — | — | — | 77.23 | — | — | — | — | 42.05 | — | — | — | 60.87 | |
| DeepSeek-V3.2Intrinsic Quality=true, Synchronic Links=true, Diachronic Links=false, Review Text=false, Data Leakage Risk=high risk2026.05 | 29.08 | — | — | — | 66.14 | — | — | — | — | 35.36 | — | — | — | 51.23 | |
| CNPE-7BModel scale=7B2026.03 | 27.89 | 37.98 | 71.96 | 68.18 | 79.53 | 57.19 | 80.84 | 54.55 | 96.75 | — | — | — | — | — | |
| CNPE-7BIntrinsic Quality=true, Synchronic Links=true, Diachronic Links=false, Review Text=false, Data Leakage Risk=low risk2026.05 | 27.39 | — | — | — | 77.32 | — | — | — | — | 39.49 | — | — | — | 59.5 | |
| DeepReview-14BParadigm=pointwise - models, Model scale=14B2026.03 | 27.19 | 31.27 | 68.17 | 12.22 | 67.75 | 21.18 | 74.24 | 42.67 | 95.78 | — | — | — | — | — | |
| AIScientistIntrinsic Quality=true, Synchronic Links=true, Diachronic Links=false, Review Text=false, Data Leakage Risk=low risk2026.05 | 25.08 | — | — | — | 74.93 | — | — | — | — | 31.62 | — | — | — | 53.5 | |
| CycleReviewer-8BIntrinsic Quality=true, Synchronic Links=true, Diachronic Links=false, Review Text=false, Data Leakage Risk=low risk2026.05 | 22.55 | — | — | — | 68.59 | — | — | — | — | 31.32 | — | — | — | 51.55 | |
| DeepReview-7BIntrinsic Quality=true, Synchronic Links=developed but not enabled, Diachronic Links=true, Review Text=false, Data Leakage Risk=low risk2026.05 | 21.1 | — | — | — | 69.38 | — | — | — | — | 29.28 | — | — | — | 49.79 | |
| PairReview(GLM)Paradigm=pairwise / listwise, Backbone=GLM2026.03 | 20.54 | 24.92 | 62.51 | 50.14 | 72.33 | 25.91 | 75.42 | 39.68 | 95.69 | — | — | — | — | — | |
| DeepReview-7BParadigm=pointwise - models, Model scale=7B2026.03 | 19.71 | 19.13 | 62.45 | 21.53 | 68.58 | 19.22 | 74.18 | 38.93 | 95.56 | — | — | — | — | — | |
| AIScientist(GPT)Paradigm=pointwise - agents, Backbone=GPT2026.03 | 19.24 | 14.29 | 63.62 | 59.75 | 74.59 | 36.84 | 77.09 | 45.74 | 95.94 | — | — | — | — | — | |
| CycleReview-7BParadigm=pointwise - models, Model scale=7B2026.03 | 18.44 | 16.99 | 62.59 | 23.44 | 66.02 | 18.28 | 70.16 | 39.89 | 95.24 | — | — | — | — | — | |
| PairReviewIntrinsic Quality=true, Synchronic Links=true, Diachronic Links=false, Review Text=false, Data Leakage Risk=low risk2026.05 | 18.23 | — | — | — | 69.78 | — | — | — | — | 26.51 | — | — | — | 50.15 | |
| PairReview(GPT)Paradigm=pairwise / listwise, Backbone=GPT2026.03 | 17.83 | 27.33 | 64.4 | 41.75 | 72.47 | 17.16 | 72.09 | 38.52 | 95.59 | — | — | — | — | — | |
| AIScientist(GLM)Paradigm=pointwise - agents, Backbone=GLM2026.03 | 16.85 | 32.24 | 30.01 | 16.08 | 68.63 | 19.66 | 75.5 | 39.97 | 95.58 | — | — | — | — | — | |
| PairReview(Gemini)Paradigm=pairwise / listwise, Backbone=Gemini2026.03 | 16.12 | 24.92 | 62.51 | 32.67 | 72.12 | 29.63 | 76.11 | 40.69 | 95.8 | — | — | — | — | — | |
| AgentReview(GLM)Paradigm=pointwise - agents, Backbone=GLM2026.03 | 15.79 | 34.38 | 44.89 | 5 | 68.95 | 15.08 | 73.56 | 39.2 | 95.46 | — | — | — | — | — | |
| Gemini-2.5-FlashIntrinsic Quality=true, Synchronic Links=true, Diachronic Links=false, Review Text=false, Data Leakage Risk=high risk2026.05 | 15.69 | — | — | — | 64.22 | — | — | — | — | 19.36 | — | — | — | 44.88 | |
| NAIPParadigm=pairwise / listwise2026.03 | 11.26 | 21.74 | 60.2 | 16.33 | 65.78 | 16.95 | 69.48 | 38.14 | 94.83 | — | — | — | — | — | |
| NAIPv1-8BIntrinsic Quality=true, Synchronic Links=false, Diachronic Links=false, Review Text=false, Data Leakage Risk=low risk2026.05 | 10.74 | — | — | — | 68.82 | — | — | — | — | 15.85 | — | — | — | 44.47 | |
| AIScientist(Gemini)Paradigm=pointwise - agents, Backbone=Gemini2026.03 | 10.6 | 28.17 | 57.76 | 9.17 | 58.36 | 11.88 | 68.63 | 35.28 | 94.62 | — | — | — | — | — | |
| SEA-EParadigm=pointwise - models2026.03 | 8.8 | 32.72 | 27.07 | 11.33 | 59.66 | 11.62 | 66.08 | 34.26 | 94.34 | — | — | — | — | — | |
| GCNIntrinsic Quality=true, Synchronic Links=true, Diachronic Links=true, Review Text=false, Data Leakage Risk=low risk2026.05 | 8.52 | — | — | — | 55.52 | — | — | — | — | 12.67 | — | — | — | 41.29 | |
| SEA-7BIntrinsic Quality=true, Synchronic Links=true, Diachronic Links=false, Review Text=false, Data Leakage Risk=low risk2026.05 | 7.54 | — | — | — | 65.85 | — | — | — | — | 9.83 | — | — | — | 41.41 | |
| AgentReview(Gemini)Paradigm=pointwise - agents, Backbone=Gemini2026.03 | 6.88 | 29.06 | 55.44 | 14 | 69.8 | 10.79 | 71.09 | 34.49 | 95.19 | — | — | — | — | — | |
| GATIntrinsic Quality=true, Synchronic Links=true, Diachronic Links=true, Review Text=false, Data Leakage Risk=low risk2026.05 | 3.49 | — | — | — | 57.34 | — | — | — | — | 4.85 | — | — | — | 37.78 | |
| GraphSAGEIntrinsic Quality=true, Synchronic Links=true, Diachronic Links=true, Review Text=false, Data Leakage Risk=low risk2026.05 | 3.48 | — | — | — | 58.68 | — | — | — | — | 4.92 | — | — | — | 37.58 | |
| AgentReview(GPT)Paradigm=pointwise - agents, Backbone=GPT2026.03 | 3.18 | 28.11 | 52.36 | 26.11 | 63.24 | 18.39 | 68.83 | 35.2 | 94.48 | — | — | — | — | — | |
| AgentReviewIntrinsic Quality=true, Synchronic Links=true, Diachronic Links=false, Review Text=false, Data Leakage Risk=low risk2026.05 | 0.26 | — | — | — | 64.11 | — | — | — | — | 0.42 | — | — | — | 37.07 | |
| AgentReview(Gemini)Evaluation Approach=pointwise - agents, Agent Provider=Gemini2026.03 | — | — | — | — | — | — | — | — | — | 10.69 | 53.56 | 10.18 | 70.82 | 58.44 | |
| AgentReview(GLM)Evaluation Approach=pointwise - agents, Agent Provider=GLM2026.03 | — | — | — | — | — | — | — | — | — | 26.43 | 58.78 | 7.29 | 69.8 | 63.63 | |
| AgentReview(GPT)Evaluation Approach=pointwise - agents, Agent Provider=GPT2026.03 | — | — | — | — | — | — | — | — | — | 5.03 | 51.87 | 23.2 | 66.74 | 55.6 | |
| AIScientist(Gemini)Evaluation Approach=pointwise - agents, Agent Provider=Gemini2026.03 | — | — | — | — | — | — | — | — | — | 16.59 | 55.62 | 9.66 | 64.64 | 60.61 | |
| AIScientist(GLM)Evaluation Approach=pointwise - agents, Agent Provider=GLM2026.03 | — | — | — | — | — | — | — | — | — | 30.29 | 58.65 | 22 | 70.91 | 60.22 | |
| AIScientist(GPT)Evaluation Approach=pointwise - agents, Agent Provider=GPT2026.03 | — | — | — | — | — | — | — | — | — | 31.06 | 60.45 | 46.37 | 78.63 | 78.79 | |
| CNPE-7BModel Scale=7B2026.03 | — | — | — | — | — | — | — | — | — | 40.91 | 64.48 | 70.76 | 81.53 | 100 | |
| CycleReviewer-8BEvaluation Approach=pointwise - models, Model Scale=8B2026.03 | — | — | — | — | — | — | — | — | — | 27.75 | 59.57 | 18.27 | 69.56 | 69.69 | |
| DeepReview-14BEvaluation Approach=pointwise - models, Model Scale=14B2026.03 | — | — | — | — | — | — | — | — | — | 40.14 | 64.19 | 14.78 | 72.04 | 82.11 | |
| DeepReview-7BEvaluation Approach=pointwise - models, Model Scale=7B2026.03 | — | — | — | — | — | — | — | — | — | 29.71 | 60.43 | 17.45 | 71.9 | 69.57 | |
| NAIPEvaluation Approach=pairwise / listwise2026.03 | — | — | — | — | — | — | — | — | — | 16.85 | 55.85 | 13.79 | 66.3 | 61.04 | |
| PairReview(Gemini)Evaluation Approach=pairwise / listwise, Agent Provider=Gemini2026.03 | — | — | — | — | — | — | — | — | — | 23.53 | 58.37 | 29.2 | 75.22 | 71.27 | |
| PairReview(GLM)Evaluation Approach=pairwise / listwise, Agent Provider=GLM2026.03 | — | — | — | — | — | — | — | — | — | 30.18 | 60.66 | 34.74 | 73.96 | 74.99 | |
| PairReview(GPT)Evaluation Approach=pairwise / listwise, Agent Provider=GPT2026.03 | — | — | — | — | — | — | — | — | — | 26.37 | 59.25 | 27.3 | 71.56 | 73.87 | |
| SEA-EEvaluation Approach=pointwise - models2026.03 | — | — | — | — | — | — | — | — | — | 13.97 | 55.05 | 12.18 | 63.05 | 50.71 |