SWE-Skills-Bench
Benchmarks
TaskDatasetSOTA resultTrendResults
SWE-Skills-Bench Hard
35Task Success Rate
20
SWE-Skills-Bench
98Baseline Accuracy
4
SWE-Skills-Bench (slo-implementation)
60Pass Rate (out of 10)
3
SWE-Skills-Bench python-anti-patterns
4Pass Rate
3
SWE-Skills-Bench implementing-agent-modes
30Pass Rate
3
SWE-Skills-Bench (changelog-automation)
60Pass Rate
3
SWE-Skills-Bench performance-optimization v3
0Pass Rate
3
SWE-Skills-Bench bazel-build-optimization
10Pass Rate
3
SWE-Skills-Bench python-resilience
30Pass Rate
3
SWE-Skills-Bench gitops-workflow
0Pass Rate
3
SWE-Skills-Bench bash-defensive-patterns
50Pass Rate
3
SWE-Skills-Bench (fix)
2Pass Rate
3
SWE-Skills-Bench python-observability
7Pass Count
3
SWE-Skills-Bench turborepo
30Pass Rate
3
SWE-Skills-Bench python-background-jobs
7Pass Rate
3
SWE-Skills-Bench service-mesh-observability
0Pass Rate
3
SWE-Skills-Bench istio-traffic-management
8Pass Rate
3
SWE-Skills-Bench (tdd-workflow)
4Pass Rate
3
SWE-Skills-Bench distributed-tracing
40Pass Rate
3
SWE-Skills-Bench similarity-search-patterns
0Pass Rate
3
SWE-Skills-Bench prompt-engineering-patterns
30Pass Rate
3
SWE-Skills-Bench (gitlab-ci-patterns)
7Pass Rate
3
SWE-Skills-Bench risk-metrics-calculation
5Pass Rate
3
SWE-Skills-Bench (Aggregate)
65Pass Rate
3