Scientific Coding on ScienceAgentBench deterministic-eval 2024
47.4Pass RateDeep negative knowledge retry
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Deep negative knowledge retryBackbone=Claude Sonnet 4.6, Number of feedback rounds=32026.06 | 47.4 | 795 | |
| Self-debugBackbone=Claude Sonnet 4.6, Number of feedback rounds=32026.06 | 44.7 | 1,109 | |
| Negative knowledge retryBackbone=Claude Sonnet 4.6, Number of feedback rounds=12026.06 | 36.8 | 296 | |
| BaseBackbone=Claude Sonnet 4.6, Protocol=First attempt2026.06 | 31.6 | — | |
| RetryBackbone=Claude Sonnet 4.6, Protocol=Fresh agent, no memory2026.06 | 31.6 | — |