Software Engineering Agent Task on SWE-Bench Pro (pass@3)
100Pass@3LLM Agent (forced-injection)
Evaluation Results
| Method | Links | |
|---|---|---|
| LLM Agent (forced-injection)Dimension=Input, Injection Condition=Inj-10, n=242026.05 | 100 | |
| LLM Agent (forced-injection)Dimension=Input, Injection Condition=Inj-30, n=242026.05 | 92 | |
| LLM Agent (forced-injection)Dimension=Context, Injection Condition=Inj-10, n=122026.05 | 92 | |
| LLM Agent (forced-injection)Dimension=Context, Injection Condition=Inj-30, n=122026.05 | 92 | |
| LLM Agent (forced-injection)Dimension=Context, Injection Condition=Inj-70, n=122026.05 | 92 | |
| LLM Agent (forced-injection)Dimension=Context, Injection Condition=Inj-90, n=122026.05 | 92 | |
| LLM Agent (forced-injection)Dimension=Goal, Injection Condition=Inj-10, n=212026.05 | 86 | |
| LLM Agent (forced-injection)Dimension=Goal, Injection Condition=Oracle, n=212026.05 | 83 | |
| LLM Agent (forced-injection)Dimension=Input, Injection Condition=Oracle, n=242026.05 | 83 | |
| LLM Agent (forced-injection)Dimension=Goal, Injection Condition=Inj-30, n=212026.05 | 81 | |
| LLM Agent (forced-injection)Dimension=Goal, Injection Condition=Inj-70, n=212026.05 | 81 | |
| LLM Agent (forced-injection)Dimension=Constraint, Injection Condition=Oracle, n=312026.05 | 81 | |
| LLM Agent (forced-injection)Dimension=Constraint, Injection Condition=Inj-10, n=312026.05 | 81 | |
| LLM Agent (forced-injection)Dimension=Input, Injection Condition=Inj-50, n=242026.05 | 79 | |
| LLM Agent (forced-injection)Dimension=Input, Injection Condition=Inj-70, n=242026.05 | 79 | |
| LLM Agent (forced-injection)Dimension=Input, Injection Condition=Inj-90, n=242026.05 | 75 | |
| LLM Agent (forced-injection)Dimension=Context, Injection Condition=NC, n=122026.05 | 75 | |
| LLM Agent (forced-injection)Dimension=Constraint, Injection Condition=Inj-30, n=312026.05 | 74 | |
| LLM Agent (forced-injection)Dimension=Constraint, Injection Condition=Inj-50, n=312026.05 | 74 | |
| LLM Agent (forced-injection)Dimension=Constraint, Injection Condition=Inj-70, n=312026.05 | 74 | |
| LLM Agent (forced-injection)Dimension=Goal, Injection Condition=Inj-90, n=212026.05 | 71 | |
| LLM Agent (forced-injection)Dimension=Constraint, Injection Condition=Inj-90, n=312026.05 | 68 | |
| LLM Agent (forced-injection)Dimension=Goal, Injection Condition=Inj-50, n=212026.05 | 67 | |
| LLM Agent (forced-injection)Dimension=Context, Injection Condition=Oracle, n=122026.05 | 67 | |
| LLM Agent (forced-injection)Dimension=Context, Injection Condition=Inj-50, n=122026.05 | 67 | |
| LLM Agent (forced-injection)Dimension=Input, Injection Condition=NC, n=242026.05 | 63 | |
| LLM Agent (forced-injection)Dimension=Constraint, Injection Condition=NC, n=312026.05 | 56 | |
| LLM Agent (forced-injection)Dimension=Goal, Injection Condition=NC, n=212026.05 | 46 |