Summarization on eBay Teammate Chat
4.47Mean Auto-Rating (r)Llama 3.2 3B
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Llama 3.2 3BVersion=r22025.11 | 4.47 | 1.44 | |
| Llama 3.1 8BVersion=r22025.11 | 4.44 | 0.495 | |
| Phi3.5mini 4BVersion=r12025.11 | 4.435 | 0.63 | |
| Llama 3.1 8BVersion=r12025.11 | 4.37 | 0.425 | |
| Qwen3 4BVersion=r22025.11 | 4.35 | 0.665 | |
| Phi3.5mini 4BVersion=r22025.11 | 4.305 | 0.5 | |
| Llama 3.2 3BVersion=r12025.11 | 4.26 | 1.23 | |
| Qwen3 4BVersion=r12025.11 | 4.15 | 0.465 | |
| Qwen3 4BVersion=out-of-box2025.11 | 4.08 | 0.395 | |
| Llama 3.2 1BVersion=r12025.11 | 4.04 | 0.565 | |
| Llama 3.2 1BVersion=r22025.11 | 4.035 | 0.56 | |
| Llama 3.1 8BVersion=out-of-box2025.11 | 3.975 | 0.03 | |
| GPT3.5Version=out-of-box2025.11 | 3.965 | — | |
| Llama 3.1 8BVersion=org2025.11 | 3.945 | — | |
| Phi3.5mini 4BVersion=org2025.11 | 3.805 | — | |
| Qwen3 4BVersion=org2025.11 | 3.685 | — | |
| Phi3.5mini 4BVersion=out-of-box2025.11 | 3.645 | -0.16 | |
| Llama 3.2 1BVersion=org2025.11 | 3.475 | — | |
| Gemma3 1BVersion=r22025.11 | 3.4 | 0.095 | |
| Gemma3 1BVersion=org2025.11 | 3.305 | — | |
| Gemma3 1BVersion=r12025.11 | 3.235 | -0.07 | |
| Llama 3.2 3BVersion=org2025.11 | 3.03 | — | |
| Gemma3 270MVersion=r22025.11 | 2.69 | 0.105 | |
| Gemma3 270MVersion=r12025.11 | 2.645 | 0.06 | |
| Gemma3 270MVersion=org2025.11 | 2.585 | — | |
| Llama 3.2 3BVersion=out-of-box2025.11 | 2.47 | -0.56 | |
| Gemma3 1BVersion=out-of-box2025.11 | 2.29 | -1.015 | |
| Llama 3.2 1BVersion=out-of-box2025.11 | 1.4 | -2.075 | |
| Gemma3 270MVersion=out-of-box2025.11 | 1.17 | -1.415 |