Limitation Generation on NeurIPS OpenReview critiques (test)
66.45CGTLimAgents
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LimAgentsBackbone=Llama 3 8B, Input=IP (All) + CP, Agents=3 Agents (Extractor, Analyzer, Reviewer), Protocol=Proposed2025.12 | 66.45 | 13.27 | 0.19 | 28.64 | 6.89 | |
| LimAgentsBackbone=GPT 4om, Input=IP (All) + CP, Agents=4 Agents, Protocol=Proposed2025.12 | 64.94 | 17.93 | 1.18 | 32.6 | 10.98 | |
| Llama 3 8B (zs)Backbone=Llama 3 8B, Input=IP (All), Agents=No Agents, Protocol=Zero-shot2025.12 | 62.04 | 14.92 | 0.37 | 29.99 | 8.73 | |
| LimAgents (Variant)Backbone=Llama 3 8B, Input=IP (All) + CP, Agents=3 Agents + zs2025.12 | 58.47 | 14.22 | 0.48 | 29.51 | 8.03 | |
| LimAgents (Variant)Backbone=GPT 4om, Input=IP (All), Agents=3 Agents2025.12 | 57.85 | 17.43 | 0.55 | 31.58 | 9.51 | |
| LimAgents (Variant)Backbone=Llama 3 8B, Input=IP (All) + CP, Agents=4 Agents (Exp 2 - Citation uses input + cited papers)2025.12 | 56.12 | 15.11 | 0.73 | 31.27 | 8.57 | |
| LimAgents (Variant)Backbone=Llama 3 8B, Input=IP (All) + CP, Agents=3 Agents + Feedback2025.12 | 53.83 | 14.02 | 0.2 | 29.67 | 7.79 | |
| LimAgents (Variant)Backbone=Llama 3 8B, Input=IP (All) + CP, Agents=4 Agents (Exp 1 - Citation uses cited papers only)2025.12 | 50.7 | 15.12 | 0.79 | 30.8 | 8.49 | |
| LimAgents (Variant)Backbone=Llama 3 8B, Input=IP (All) + CP, Agents=9 Agents2025.12 | 49.81 | 16.05 | 0.69 | 31.52 | 8.89 | |
| GPT 4om (zs)Backbone=GPT 4om, Input=IP (All), Agents=No Agents, Protocol=Zero-shot2025.12 | 49.43 | 18.83 | 0.6 | 38.87 | 10.38 | |
| GPT 4om (zs)Backbone=GPT 4om, Input=IP (3 sec), Agents=No Agents, Protocol=Zero-shot2025.12 | 45.69 | 17.07 | 0.4 | 35.42 | 9.12 |