Multi-turn Dialogue Reasoning on MuTual (test)
84.6MRRFOCAL REASONER
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| FOCAL REASONER2021.05 | 84.6 | 72.7 | 91 | — | |
| ROBERTaScoring method=default individual scoring method2021.05 | 83.6 | 71.3 | 89.2 | — | |
| ROBERTa-MCScoring method=multi-choice method2021.05 | 82.2 | 68.6 | 88.7 | — | |
| SPIDERBackbone=ELECTRA-large, Objective=Full SPIDER2021.05 | 0.956 | 0.92 | 0.979 | — | |
| SBRBackbone=ELECTRA-large, Objective=Sentence Backbone Reconstruction2021.05 | 0.955 | 0.916 | 0.978 | — | |
| UORBackbone=ELECTRA-large, Objective=Utterance Order Restoration2021.05 | 0.953 | 0.913 | 0.978 | — | |
| ELECTRA-largeBackbone=ELECTRA-large2021.05 | 0.949 | 0.906 | 0.977 | — | |
| SPIDERBackbone=ELECTRA-base, Objective=Full SPIDER2021.05 | 0.882 | 0.792 | 0.923 | — | |
| SBRBackbone=ELECTRA-base, Objective=Sentence Backbone Reconstruction2021.05 | 0.876 | 0.771 | 0.92 | — | |
| UORBackbone=ELECTRA-base, Objective=Utterance Order Restoration2021.05 | 0.869 | 0.766 | 0.918 | — | |
| ELECTRA-baseBackbone=ELECTRA-base2021.05 | 0.865 | 0.762 | 0.916 | — | |
| SPIDERBackbone=BERT-large, Objective=Full SPIDER2021.05 | 0.839 | 0.718 | 0.892 | — | |
| SBRBackbone=BERT-large, Objective=Sentence Backbone Reconstruction2021.05 | 0.834 | 0.71 | 0.894 | — | |
| UORBackbone=BERT-large, Objective=Utterance Order Restoration2021.05 | 0.828 | 0.698 | 0.886 | — | |
| BERT-largeBackbone=BERT-large2021.05 | 0.822 | 0.691 | 0.879 | — | |
| SPIDERBackbone=BERT-base, Objective=Full SPIDER2021.05 | 0.816 | 0.676 | 0.873 | — | |
| SBRBackbone=BERT-base, Objective=Sentence Backbone Reconstruction2021.05 | 0.813 | 0.674 | 0.871 | — | |
| UORBackbone=BERT-base, Objective=Utterance Order Restoration2021.05 | 0.807 | 0.661 | 0.867 | — | |
| BERT-baseBackbone=BERT-base2021.05 | 0.8 | 0.653 | 0.86 | — | |
| AioliModel Size=500M, Base Model=Qwen-2, Training Steps=2000, Batch Size=32, Context Length=512, K=20, E=102026.06 | — | — | — | 73.57 | |
| DoGEModel Size=500M, Base Model=Qwen-2, Training Steps=2000, Batch Size=32, Context Length=512, K=20, E=102026.06 | — | — | — | 71.67 | |
| DoReMiModel Size=500M, Base Model=Qwen-2, Training Steps=2000, Batch Size=32, Context Length=512, K=20, E=102026.06 | — | — | — | 70.97 | |
| Skill-ItModel Size=500M, Base Model=Qwen-2, Training Steps=2000, Batch Size=32, Context Length=512, K=20, E=102026.06 | — | — | — | 73.07 | |
| TANDEMModel Size=500M, Base Model=Qwen-2, Training Steps=2000, Batch Size=32, Context Length=512, K=20, E=102026.06 | — | — | — | 73.77 | |
| UniformModel Size=500M, Base Model=Qwen-2, Training Steps=2000, Batch Size=32, Context Length=512, K=20, E=102026.06 | — | — | — | 72.72 |