Issue Resolution on tinydb GitHub Issues OSS repo (10 randomly selected)
6Issues ResolvedClaude 4 Sonnet
Evaluation results
| Method | Links | |
|---|---|---|
| Claude 4 Sonnetwall-clock budget=same, tool budget=same2025.12 | 6 | |
| Gemini-2.5 prowall-clock budget=same, tool budget=same2025.12 | 6 | |
| AIRwall-clock budget=same, tool budget=same2025.12 | 5 |