Software Engineering Issue Resolving on SWE-bench Verified
66Pass@1Context Weaver
Evaluation Results
| Method | Links | |
|---|---|---|
| Context WeaverModel=Claude Sonnet 4, Setting=Unified2026.04 | 66 | |
| LLM SummarizationModel=Claude Sonnet 4, Setting=Unified2026.04 | 64.2 | |
| MTL (I)Model Backbone=GPT-5-mini2026.04 | 63.3 | |
| Sliding WindowModel=Claude Sonnet 4, Setting=Unified, window size=52026.04 | 63.2 | |
| ZeroShotModel Backbone=GPT-5-mini2026.04 | 62.3 | |
| MTL (W)Model Backbone=GPT-5-mini2026.04 | 62 | |
| MTL (S)Model Backbone=GPT-5-mini2026.04 | 61.3 | |
| MTL (T)Model Backbone=GPT-5-mini2026.04 | 61 | |
| Sliding WindowModel=Gemini 3 Flash, Setting=Unified, window size=52026.04 | 60.4 | |
| Context WeaverModel=GPT-5, Setting=Hybrid2026.04 | 58.6 | |
| Context WeaverModel=Gemini 3 Flash, Setting=Unified2026.04 | 58.4 | |
| Sliding WindowModel=GPT-5, Setting=Unified, window size=52026.04 | 57.4 | |
| Context WeaverModel=GPT-5, Setting=Unified2026.04 | 56.8 | |
| LLM SummarizationModel=Gemini 3 Flash, Setting=Unified2026.04 | 56.8 | |
| LLM SummarizationModel=GPT-5, Setting=Unified2026.04 | 46.7 | |
| MTL (I)Model Backbone=DeepSeek V3.22026.04 | 46.3 | |
| ZeroShotModel Backbone=DeepSeek V3.22026.04 | 42.3 | |
| MTL (I)Model Backbone=Qwen3-Coder-480B-A35B-Instruct2026.04 | 37 | |
| ZeroShotModel Backbone=Qwen3-Coder-480B-A35B-Instruct2026.04 | 34.7 |