Experimental reproduction on REPRODUCEBENCH
91.57Align-Score (Paper-Level)AUTO-REPRODUCE
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| AUTO-REPRODUCELLM=Gemini-2.5-Pro2025.05 | 91.57 | 60.26 | 77.56 | 94.87 | 19.72 | |
| AUTO-REPRODUCELLM=o3-mini2025.05 | 90.86 | 58.48 | 75.21 | 92.31 | 24.31 | |
| PaperCoderLLM=o3-mini2025.05 | 90.41 | 47.54 | 60.26 | 17.94 | 89.23 | |
| AUTO-REPRODUCELLM=Claude-3.5-Sonnet2025.05 | 90.27 | 54.11 | 69.97 | 84.62 | 31.62 | |
| AUTO-REPRODUCELLM=GPT-4o2025.05 | 82.13 | 41.52 | 56.24 | 76.92 | 41.77 | |
| Agent LaboratoryLLM=GPT-4o2025.05 | 63.47 | 35.32 | 48.64 | 23.08 | 82.31 | |
| ChatDevLLM=GPT-4o2025.05 | 57.33 | 32.8 | 43.33 | 2.56 | 99.62 |