Tool Use Accuracy on Unseen Tools
1SRtChatHuman
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ChatHuman2024.05 | 1 | 0.968 | 0.893 | 0.953 | 0.953 | |
| ChatHumanBackbone=LLaVA-1.5-7B2024.05 | 0.999 | 0.967 | 0.893 | 0.954 | 0.953 | |
| Visual ChatGPT-4Model Version=GPT-42024.05 | 0.998 | 0.913 | 0.801 | 0.872 | 0.907 | |
| Ours w/ GPT-4LLM Backbone=GPT-42024.05 | 0.969 | 0.924 | 0.734 | 0.746 | 0.876 | |
| GPT4Tools-FTFine-tuned=true2024.05 | 0.904 | 0.807 | 0.69 | 0.747 | 0.8 | |
| GPT4ToolsFine-tuned=false2024.05 | 0.612 | 0.546 | 0.542 | 0.525 | 0.573 | |
| Visual ChatGPT-3.5Model Version=GPT-3.52024.05 | 0.507 | 0.314 | 0.226 | 0.293 | 0.803 |