Code Debugging on MBPP
80.8AccuracyMGDebugger
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MGDebuggerBackbone=CodeQwen1.52024.10 | 80.8 | 41.1 | |
| MGDebuggerBackbone=DeepSeek-Coder-V2-Lite2024.10 | 80 | 39 | |
| ReflexionBackbone=CodeQwen1.52024.10 | 78.6 | 34.4 | |
| Self-Debugging (Expl.)Backbone=CodeQwen1.52024.10 | 77.4 | 30.7 | |
| Self-Debugging (Trace)Backbone=CodeQwen1.52024.10 | 76.8 | 28.8 | |
| MGDebuggerBackbone=Codestral2024.10 | 76.8 | 32.9 | |
| ReflexionBackbone=DeepSeek-Coder-V2-Lite2024.10 | 76.6 | 28.7 | |
| Self-Debugging (Expl.)Backbone=Codestral2024.10 | 76.4 | 31.8 | |
| Self-EditBackbone=Codestral2024.10 | 75.8 | 30 | |
| ReflexionBackbone=Codestral2024.10 | 75.2 | 28.3 | |
| Self-EditBackbone=CodeQwen1.52024.10 | 75 | 23.3 | |
| LDB (Block)Backbone=DeepSeek-Coder-V2-Lite2024.10 | 74 | 20.7 | |
| Simple FeedbackBackbone=CodeQwen1.52024.10 | 74 | 20.2 | |
| Self-Debugging (Trace)Backbone=Codestral2024.10 | 73.6 | 23.7 | |
| Self-Debugging (Expl.)Backbone=DeepSeek-Coder-V2-Lite2024.10 | 73.4 | 18.9 | |
| LDB (Block)Backbone=CodeQwen1.52024.10 | 72.8 | 16.6 | |
| LDB (Function)Backbone=CodeQwen1.52024.10 | 72.8 | 16.6 | |
| LDB (Function)Backbone=DeepSeek-Coder-V2-Lite2024.10 | 72.6 | 16.5 | |
| Self-Debugging (Trace)Backbone=DeepSeek-Coder-V2-Lite2024.10 | 72.6 | 16.5 | |
| LDB (Line)Backbone=CodeQwen1.52024.10 | 72.6 | 16 | |
| LDB (Block)Backbone=Codestral2024.10 | 72.2 | 19.7 | |
| LDB (Function)Backbone=Codestral2024.10 | 72 | 19.1 | |
| LDB (Line)Backbone=DeepSeek-Coder-V2-Lite2024.10 | 71.8 | 14 | |
| LDB (Line)Backbone=Codestral2024.10 | 71.8 | 18.5 | |
| Simple FeedbackBackbone=Codestral2024.10 | 71.6 | 17.9 | |
| Self-EditBackbone=DeepSeek-Coder-V2-Lite2024.10 | 71.2 | 12.2 | |
| Simple FeedbackBackbone=DeepSeek-Coder-V2-Lite2024.10 | 69.4 | 6.7 | |
| No-DebuggingBackbone=CodeQwen1.52024.10 | 67.4 | — | |
| No-DebuggingBackbone=DeepSeek-Coder-V2-Lite2024.10 | 67.2 | — | |
| No-DebuggingBackbone=Codestral2024.10 | 65.4 | — |