Dialogue Modeling on MSC (Multi-Session Chat) 1.0 (test)
8.2Session 1 PerplexitySumMem-MSC 2.7B (FiD)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SumMem-MSC 2.7B (FiD)Parameters=2.7B, Retrieval Strategy=FiD2021.07 | 8.2 | 8.71 | 8.89 | 9 | 9.07 | 7.91 | |
| MSC 2.7B (RAG)Parameters=2.7B, Retrieval Strategy=RAG2021.07 | 8.22 | 8.78 | 8.97 | 9.11 | 9.17 | 8.1 | |
| MSC 2.7B (FiD)Parameters=2.7B, Retrieval Strategy=FiD2021.07 | 8.22 | 8.75 | 8.92 | 9.05 | 9.11 | 8.06 | |
| SumMem-MSC 2.7B (FiD-RAG)Parameters=2.7B, Retrieval Strategy=FiD-RAG2021.07 | 8.22 | 8.7 | 8.89 | 9 | 9.07 | 7.87 | |
| MSC 2.7B (FiD-RAG)Parameters=2.7B, Retrieval Strategy=FiD-RAG2021.07 | 8.23 | 8.75 | 8.93 | 9.04 | 9.11 | 8.03 | |
| SumMem-MSC 2.7B (RAG)Parameters=2.7B, Retrieval Strategy=RAG2021.07 | 8.24 | 8.81 | 9 | 9.1 | 9.17 | 8.05 | |
| MSC 2.7BParameters=2.7B, Context Truncation=10242021.07 | 8.25 | 8.76 | 8.93 | 9.07 | 9.16 | 8.09 | |
| SumMem-MSC 2.7BParameters=2.7B, Context Truncation=10242021.07 | 8.25 | 8.71 | 8.89 | 9.01 | 9.09 | 8.04 | |
| MSC 2.7BParameters=2.7B, Context Truncation=1282021.07 | 8.87 | 8.89 | 9.1 | 9.21 | 9.27 | 8.95 | |
| BST 2.7BParameters=2.7B2021.07 | 8.97 | 9.98 | 10.26 | 10.4 | 10.5 | 12.92 |