Agent Task-solving on OpenAct 1.0 (test)
68.9Finance Task Success RateOpenAgent
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| OpenAgentBackbone=GPT-42023.12 | 68.9 | 34.9 | 86.7 | 45.6 | 16.7 | 43.3 | 35.4 | 47.3 | |
| ReActBackbone=GPT-42023.12 | 51.1 | 19.7 | 17.8 | 22.2 | 10.4 | 30 | 23.3 | 24.6 | |
| ReAct + Sum.Backbone=GPT-42023.12 | 31.1 | 19.7 | 26.7 | 22.9 | 14.8 | 33.3 | 26.7 | 24.4 | |
| OpenAgentBackbone=GPT-3.5-Turbo2023.12 | 8.9 | 24.2 | 23.3 | 8.9 | 10 | 33.3 | 20.1 | 17.1 | |
| ReActBackbone=GPT-3.5-Turbo2023.12 | 2.2 | 3 | 3.3 | 6.7 | 0 | 0 | 0 | 2.4 | |
| VanillaBackbone=GPT-3.5-Turbo2023.12 | 0 | 36.4 | 0 | 0 | 0 | 0 | 31.3 | 11.5 | |
| ReAct + Sum.Backbone=GPT-3.5-Turbo2023.12 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| VanillaBackbone=GPT-42023.12 | 0 | 68.2 | 0 | 0 | 0 | 0 | 43.8 | 19.5 | |
| XAgentBackbone=GPT-42023.12 | 0 | 40.9 | 0 | 0 | 40 | 0 | 81.3 | 23 |