Response Generation on Ubuntu IRC
17.81BLEU-1Qwen3-8B+DRCR
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Qwen3-8B+DRCRModel Category=LLM-based MDG, Backbone=Qwen3-8B, Training Protocol=DRCR2026.04 | 17.81 | 8.02 | 5.79 | 4.26 | 8.49 | 14.82 | |
| Qwen3-4B+DRCRModel Category=LLM-based MDG, Backbone=Qwen3-4B, Training Protocol=DRCR2026.04 | 16.95 | 7.51 | 5.32 | 3.94 | 8.03 | 13.55 | |
| Llama3.2-3B+DRCRModel Category=LLM-based MDG, Backbone=Llama3.2-3B, Training Protocol=DRCR2026.04 | 16.91 | 7.24 | 4.95 | 3.71 | 7.92 | 13.37 | |
| SS-MPCModel Category=SLM-based MDG2026.04 | 15.6 | 6.62 | 3.67 | 2.44 | 7.75 | 12.44 | |
| Qwen3-8B+SFTModel Category=LLM-based MDG, Backbone=Qwen3-8B, Training Protocol=SFT2026.04 | 14.22 | 6.53 | 3.83 | 2.52 | 6.37 | 12.92 | |
| RL-TRCModel Category=SLM-based MDG2026.04 | 13.66 | 6.58 | 4.1 | 2.93 | 6.2 | 12.72 | |
| Qwen3-4B+SFTModel Category=LLM-based MDG, Backbone=Qwen3-4B, Training Protocol=SFT2026.04 | 13.48 | 6.05 | 3.54 | 2.39 | 6.14 | 12.2 | |
| Llama3.2-3B+SFTModel Category=LLM-based MDG, Backbone=Llama3.2-3B, Training Protocol=SFT2026.04 | 13.23 | 5.87 | 3.37 | 2.2 | 6.02 | 11.94 | |
| MADNetModel Category=SLM-based MDG2026.04 | 12.73 | 5.12 | 2.64 | 1.63 | 5.31 | 11.74 | |
| HeterMPC_BERT2023.05 | 12.61 | 4.55 | 2.25 | 1.41 | 4.79 | 11.2 | |
| EMMDGModel Category=SLM-based MDG2026.04 | 12.31 | 5.39 | 3.34 | 2.45 | 5.52 | 11.71 | |
| HeterMPCModel Category=SLM-based MDG2026.04 | 12.26 | 4.8 | 2.42 | 1.49 | 4.94 | 11.2 | |
| BERT-our2023.05 | 11.78 | 4.74 | 2.71 | 1.96 | 5.09 | 11.21 | |
| BERT2023.05 | 10.9 | 3.85 | 1.69 | 0.89 | 4.18 | 9.8 | |
| GSN2023.05 | 10.23 | 3.57 | 1.7 | 0.97 | 4.1 | 9.91 | |
| GSNModel Category=SLM-based MDG2026.04 | 10.23 | 3.57 | 1.7 | 0.97 | 4.1 | 9.91 |