Dialogue Generation on PERSUASIVETOM
80.2ToM Score (Judge: Llama-3.3-70B)COSTOM
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| COSTOMBase Model=Qwen2.5-7B-Instruct, Category=Tuning & Intervention2026.04 | 80.2 | 81.1 | 71.3 | 74.6 | 88.8 | 81.2 | 76 | 84 | 79.5 | |
| COSTOMBase Model=Llama-3-8B-Instruct, Category=Tuning & Intervention2026.04 | 79.7 | 81.1 | 75.7 | 70.3 | 80.7 | 74.4 | 71.5 | 80.5 | 74 | |
| LatentQABase Model=Qwen2.5-7B-Instruct, Category=Tuning & Intervention2026.04 | 67.1 | 75.1 | 68.1 | 63 | 77.3 | 63.8 | 64 | 76.5 | 64.5 | |
| Full-Layer LoRABase Model=Qwen2.5-7B-Instruct, Category=Tuning & Intervention2026.04 | 65.4 | 80.2 | 63.6 | 34.3 | 81.7 | 67.8 | 44.5 | 79.5 | 66 | |
| MindDial (Fine-tuned)Base Model=Llama-3-8B-Instruct, Category=Tuning & Intervention2026.04 | 64.3 | 72 | 64.5 | 61 | 74.7 | 60 | 59 | 68.5 | 61.5 | |
| MindDial (Fine-tuned)Base Model=Qwen2.5-7B-Instruct, Category=Tuning & Intervention2026.04 | 64.3 | 72 | 64.5 | 64.5 | 72.3 | 67 | 63.5 | 71.5 | 65.5 | |
| Full-Layer LoRABase Model=Llama-3-8B-Instruct, Category=Tuning & Intervention2026.04 | 62.8 | 79.8 | 65.1 | 36.7 | 86.4 | 72.9 | 48.5 | 79 | 67 | |
| MindDial (prompt)Base Model=Llama-3-8B-Instruct, Category=Prompting Baselines2026.04 | 45.1 | 55.4 | 48.5 | 37.4 | 47.8 | 37.7 | 38.5 | 53.5 | 42 | |
| MindDial (prompt)Base Model=Qwen2.5-7B-Instruct, Category=Prompting Baselines2026.04 | 44.2 | 56 | 45.5 | 58.6 | 66.7 | 59.6 | 49.5 | 61 | 51.5 | |
| LatentQABase Model=Llama-3-8B-Instruct, Category=Tuning & Intervention2026.04 | 25.5 | 32.4 | 26.9 | 30 | 50.4 | 29.8 | 27.5 | 44 | 28.5 | |
| Zero-shot BaselineBase Model=Qwen2.5-7B-Instruct, Category=Prompting Baselines2026.04 | 10.9 | 43.1 | 19 | 22.7 | 78.8 | 52.2 | 14.5 | 51.5 | 30.5 | |
| Zero-shot BaselineBase Model=Llama-3-8B-Instruct, Category=Prompting Baselines2026.04 | 10.4 | 43.3 | 19.9 | 25.9 | 78.3 | 52.9 | 16.5 | 52 | 31 |