Agentic Task Completion on AppWorld (test-normal)
56.5AccuracyACON UTCO
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ACON UTCOAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=History Compression2025.10 | 56.5 | 22.82 | 7.33 | 4.69 | |
| ACON UTCOAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=History Compression2025.10 | 56.2 | — | 7.48 | 4.43 | |
| No compressionAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=None2025.10 | 56 | 16.14 | 9.93 | 5.96 | |
| ACON UTCOAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=Observation Compression2025.10 | 53.6 | 18.12 | 7.43 | 4.93 | |
| ACON UTAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=History Compression2025.10 | 51.2 | 20.92 | 7.17 | 4.49 | |
| ACON UTAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=History Compression2025.10 | 50 | — | 7.39 | 4.47 | |
| No compressionAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=None2025.10 | 47.9 | — | 10.1 | 5.36 | |
| ACON UTCOAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=Observation Compression2025.10 | 47.9 | — | 7.3 | 4.43 | |
| ACON UTAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=Observation Compression2025.10 | 47 | 16.67 | 7.62 | 5.08 | |
| FIFOAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=History Compression2025.10 | 45.8 | 28.48 | 6.73 | 5.69 | |
| ACON UTAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=Observation Compression2025.10 | 45.8 | — | 7.79 | 5 | |
| PromptingAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=History Compression2025.10 | 43.5 | 24.01 | 6.93 | 5.29 | |
| PromptingAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=Observation Compression2025.10 | 42.3 | 17.38 | 6.58 | 4.09 | |
| PromptingAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=History Compression2025.10 | 41.7 | — | 7.1 | 5.36 | |
| FIFOAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=History Compression2025.10 | 39.6 | — | 7.26 | 6.24 | |
| LLMLinguaAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=History Compression2025.10 | 39.3 | 24.42 | 7.5 | 6.37 | |
| LLMLinguaAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=History Compression2025.10 | 37.5 | — | 8.04 | 7.39 | |
| PromptingAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=Observation Compression2025.10 | 35.4 | — | 6.96 | 4.11 | |
| LLMLinguaAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=Observation Compression2025.10 | 32.1 | 18.16 | 8.17 | 6.01 | |
| LLMLinguaAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=Observation Compression2025.10 | 29.2 | — | 8.24 | 5.23 | |
| RetrievalAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=History Compression2025.10 | 27.4 | 33.17 | 8.39 | 6.68 | |
| RetrievalAgent=gpt-4.1, Compressor=gpt-4.1, Compression Type=History Compression2025.10 | 12.5 | — | 8.74 | 7.72 |