Dialogue Agent Interaction on DEMO
8.447Goal AchievementQwen2-72B-Instruct
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Qwen2-72B-InstructModel Category=Open-sourced LLM, Temperature=12024.12 | 8.447 | 9.204 | 8.699 | 8.175 | 8.631 | 6.608 | |
| GPT-4oModel Category=Proprietary LLM, Temperature=12024.12 | 8.19 | 9.181 | 8.614 | 8.537 | 8.631 | 6.793 | |
| Claude-3.5-SonnetModel Category=Proprietary LLM, Temperature=12024.12 | 7.571 | 9.174 | 8.432 | 8.84 | 8.504 | 6.599 | |
| GPT-4o-miniModel Category=Proprietary LLM, Temperature=12024.12 | 7.551 | 9.082 | 8.316 | 8.163 | 8.278 | 6.449 | |
| DEMO-Qwen2-7BModel Category=DEMO Agent, Backbone=Qwen2-7B-Instruct, Temperature=12024.12 | 7.45 | 8.864 | 8.073 | 7.864 | 8.063 | 6.625 | |
| Claude-3.5-HaikuModel Category=Proprietary LLM, Temperature=12024.12 | 7.361 | 9.115 | 8.402 | 7.968 | 8.212 | 6.358 | |
| Qwen2-72B-InstructModel Category=Open-sourced LLM, Evaluator=Gemini-1.5-Pro2024.12 | 7.335 | 6.213 | 6.967 | 5.404 | 6.482 | — | |
| Llama-3.1-70B-InstructModel Category=Open-sourced LLM, Temperature=12024.12 | 7.223 | 7.914 | 7.222 | 6.753 | 7.278 | 6.154 | |
| GPT-4oModel Category=Proprietary LLM, Evaluator=Gemini-1.5-Pro2024.12 | 7.215 | 7.032 | 7.942 | 6.328 | 7.132 | — | |
| DEMO-Llama3.1-8BModel Category=DEMO Agent, Backbone=Llama3.1-8B-Instruct, Temperature=12024.12 | 6.945 | 7.688 | 7.015 | 6.378 | 7.707 | 6.341 | |
| Qwen2-7B-InstructModel Category=Backbone LLM, Temperature=12024.12 | 6.698 | 8.112 | 6.895 | 6.278 | 6.996 | 5.828 | |
| Claude-3.5-SonnetModel Category=Proprietary LLM, Evaluator=Gemini-1.5-Pro2024.12 | 6.55 | 7.284 | 7.664 | 6.526 | 7.006 | — | |
| DEMO-Qwen2-7BModel Category=DEMO Agent, Evaluator=Gemini-1.5-Pro2024.12 | 6.306 | 6.287 | 7 | 5.586 | 6.296 | — | |
| Llama-3.1-70B-InstructModel Category=Open-sourced LLM, Evaluator=Gemini-1.5-Pro2024.12 | 6.25 | 5.163 | 5.898 | 4.461 | 5.45 | — | |
| GPT-4o-miniModel Category=Proprietary LLM, Evaluator=Gemini-1.5-Pro2024.12 | 6.159 | 6.67 | 7.234 | 5.944 | 6.503 | — | |
| Claude-3.5-HaikuModel Category=Proprietary LLM, Evaluator=Gemini-1.5-Pro2024.12 | 6.049 | 6.719 | 7.268 | 5.742 | 6.446 | — | |
| DEMO-Llama3.1-8BModel Category=DEMO Agent, Evaluator=Gemini-1.5-Pro2024.12 | 5.859 | 4.814 | 5.549 | 3.948 | 5.051 | — | |
| Llama3.1-8B-InstructModel Category=Backbone LLM, Temperature=12024.12 | 5.831 | 6.166 | 5.519 | 4.974 | 5.623 | 5.335 | |
| Qwen2-7B-InstructModel Category=Backbone LLM, Evaluator=Gemini-1.5-Pro2024.12 | 5.528 | 5.274 | 5.043 | 4.209 | 5.018 | — | |
| Llama3.1-8B-InstructModel Category=Backbone LLM, Evaluator=Gemini-1.5-Pro2024.12 | 4.784 | 4.195 | 4.536 | 3.573 | 4.274 | — |