Error Propagation Analysis on AgentProp-Bench semantic-wrong 1,350 traces (P2)
74S1 (%)GPT-4o
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GPT-4on=50, nS2=112026.04 | 74 | 22 | 74 | 0.3 | 0.82 | 0.68 | |
| o3-minin=50, nS2=82026.04 | 74 | 16 | 70 | 0.22 | 0.62 | 0.76 | |
| GPT-4.1-nanon=200, nS2=352026.04 | 62.5 | 17.5 | 84.5 | 0.28 | 0.97 | 0.83 | |
| DeepSeek-V3n=200, nS2=142026.04 | 52 | 7 | 62.5 | 0.13 | 0.43 | 0.57 | |
| GPT-4.1-minin=200, nS2=212026.04 | 50.5 | 10.5 | 65.5 | 0.21 | 0.67 | 0.66 | |
| Gemini-2.5-Fn=50, nS2=32026.04 | 50 | 6 | 76 | 0.12 | 0.67 | 0.84 | |
| GPT-4o-minin=200, nS2=452026.04 | 40 | 22.5 | 71 | 0.56 | 0.84 | 0.84 | |
| GPT-3.5-Tn=200, nS2=192026.04 | 37.5 | 9.5 | 67.5 | 0.25 | 0.74 | 0.76 | |
| Gemini-2.0-Fn=200, nS2=22026.04 | 5 | 1 | 67 | 0.2 | 0.5 | 0.9 |