Automated Scientific Research Evaluation on SciSafetyBench
0Reject RateAI Scientist
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| AI ScientistModel=Llama-3.1-70B2026.06 | 0 | 1.85 | 1.9 | 1.9 | 1.9 | 3.2 | 2.45 | |
| AI ScientistModel=Qwen2.5-72B2026.06 | 0 | 1.88 | 1.93 | 1.92 | 1.92 | 3.25 | 2.48 | |
| AI ScientistModel=GPT-4o2026.06 | 0 | 2 | 2.1 | 2.05 | 2 | 3.4 | 2.6 | |
| AI ScientistModel=DeepSeek-V32026.06 | 2 | 1.9 | 1.95 | 1.95 | 1.93 | 3.28 | 2.5 | |
| ResearchTownModel=Llama-3.1-70B2026.06 | 3 | 1.92 | 1.97 | 1.95 | 1.88 | 3.17 | 2.4 | |
| ResearchTownModel=Qwen2.5-72B2026.06 | 5 | 1.95 | 2 | 1.98 | 1.9 | 3.2 | 2.43 | |
| ResearchTownModel=DeepSeek-V32026.06 | 5 | 1.97 | 2.02 | 2 | 1.92 | 3.22 | 2.45 | |
| ResearchTownModel=GPT-4o2026.06 | 5 | 2.05 | 2.12 | 2.1 | 2 | 3.35 | 2.55 | |
| CycleResearcherModel=Llama-3.1-70B2026.06 | 8 | 1.98 | 2.05 | 2.02 | 1.93 | 3.28 | 2.53 | |
| CycleResearcherModel=Qwen2.5-72B2026.06 | 10 | 2 | 2.08 | 2.05 | 1.95 | 3.3 | 2.57 | |
| CycleResearcherModel=DeepSeek-V32026.06 | 10 | 2.02 | 2.1 | 2.08 | 1.97 | 3.32 | 2.6 | |
| AI Co-ScientistModel=Llama-3.1-70B2026.06 | 12 | 2.05 | 2.18 | 2.12 | 2.02 | 3.38 | 2.68 | |
| CycleResearcherModel=GPT-4o2026.06 | 12 | 2.12 | 2.22 | 2.18 | 2.05 | 3.45 | 2.72 | |
| Agent LaboratoryModel=Llama-3.1-70B2026.06 | 15 | 2 | 2.47 | 2.47 | 1.94 | 3.18 | 2.45 | |
| AI Co-ScientistModel=Qwen2.5-72B2026.06 | 15 | 2.08 | 2.22 | 2.15 | 2.05 | 3.42 | 2.72 | |
| AI Co-ScientistModel=DeepSeek-V32026.06 | 17 | 2.1 | 2.25 | 2.18 | 2.08 | 3.45 | 2.75 | |
| Agent LaboratoryModel=Qwen2.5-72B2026.06 | 18 | 2.02 | 2.5 | 2.5 | 1.97 | 3.22 | 2.5 | |
| Agent LaboratoryModel=DeepSeek-V32026.06 | 20 | 2.05 | 2.52 | 2.52 | 1.98 | 3.25 | 2.52 | |
| AI Co-ScientistModel=GPT-4o2026.06 | 20 | 2.18 | 2.35 | 2.28 | 2.12 | 3.55 | 2.85 | |
| Agent LaboratoryModel=GPT-4o2026.06 | 22 | 2.12 | 2.6 | 2.6 | 2.05 | 3.35 | 2.58 | |
| SafeScientistModel=Llama-3.1-70B2026.06 | 85 | 2 | 2.48 | 2.5 | 1.98 | 3.47 | 4.72 | |
| SafeScientistModel=Qwen2.5-72B2026.06 | 87 | 2.02 | 2.5 | 2.52 | 2 | 3.5 | 4.75 | |
| SafeScientistModel=DeepSeek-V32026.06 | 88 | 2.05 | 2.52 | 2.55 | 2.02 | 3.52 | 4.78 | |
| SafeScientistModel=GPT-4o2026.06 | 90 | 2.1 | 2.62 | 2.65 | 2.1 | 3.62 | 4.83 | |
| SciTraceModel=Llama-3.1-70B2026.06 | 92 | 2.12 | 2.62 | 2.58 | 2.1 | 3.68 | 4.87 | |
| SciTraceModel=Qwen2.5-72B2026.06 | 93 | 2.15 | 2.65 | 2.62 | 2.12 | 3.72 | 4.89 | |
| SciTraceModel=DeepSeek-V32026.06 | 94 | 2.18 | 2.68 | 2.65 | 2.15 | 3.75 | 4.91 | |
| SciTraceModel=GPT-4o2026.06 | 95 | 2.22 | 2.75 | 2.72 | 2.2 | 3.82 | 4.93 |