# Multi-turn Dialogue on MT-Bench (MAT and Speedup) Source: [Multi-turn Dialogue on MT-Bench (MAT and Speedup)]() Collection index: [llms.txt]() Data updated: 2026-05-20T09:47:02.411Z Evaluation of multi-turn dialogue performance on the MT-Bench dataset, focusing on response quality (MAT) and system efficiency (Speedup). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multi-turn Dialogue | | Dataset (including any stated split) | MT-Bench | | Metric | MAT Score | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 30 of 30 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [MAT Score](): Higher is better; unit: SCALAR. - [Speedup](): Higher is better; unit: SCALAR. ## Recorded results | Method | MAT Score | Result date | Report paper | | --- | --- | --- | --- | | Graft | 6.93 | Not recorded | 6a0d405cf984739ed6737b93 | | EAGLE3-2K | 6.81 | Not recorded | 6a0d405cf984739ed6737b93 | | Graft | 6.49 | Not recorded | 6a0d405cf984739ed6737b93 | | EAGLE3-2K | 6.42 | Not recorded | 6a0d405cf984739ed6737b93 | | ECHO | 6.35 | Not recorded | 6a0d405cf984739ed6737b93 | | ECHO | 6.23 | Not recorded | 6a0d405cf984739ed6737b93 | | DDD | 6.07 | Not recorded | 6a0d405cf984739ed6737b93 | | Graft-DFLASH(16) | 4.54 | Not recorded | 6a0d405cf984739ed6737b93 | | DFLASH(16) | 4.39 | Not recorded | 6a0d405cf984739ed6737b93 | | Graft | 3.79 | Not recorded | 6a0d405cf984739ed6737b93 | | EAGLE3 | 3.71 | Not recorded | 6a0d405cf984739ed6737b93 | | ECHO | 3.68 | Not recorded | 6a0d405cf984739ed6737b93 | | Graft-EAGLE3(60) | 3.41 | Not recorded | 6a0d405cf984739ed6737b93 | | EAGLE3(60) | 3.34 | Not recorded | 6a0d405cf984739ed6737b93 | | Graft | 3.15 | Not recorded | 6a0d405cf984739ed6737b93 | | EAGLE3 | 3.02 | Not recorded | 6a0d405cf984739ed6737b93 | | ECHO | 2.96 | Not recorded | 6a0d405cf984739ed6737b93 | | Medusa | 2.61 | Not recorded | 6a0d405cf984739ed6737b93 | | Graft | 2.61 | Not recorded | 6a0d405cf984739ed6737b93 | | EAGLE3-2K | 2.56 | Not recorded | 6a0d405cf984739ed6737b93 | | ECHO | 2.38 | Not recorded | 6a0d405cf984739ed6737b93 | | TR | 2.37 | Not recorded | 6a0d405cf984739ed6737b93 | | TR | 2.37 | Not recorded | 6a0d405cf984739ed6737b93 | | TR | 2.36 | Not recorded | 6a0d405cf984739ed6737b93 | | Sps | 2.26 | Not recorded | 6a0d405cf984739ed6737b93 | | SAMD | 2.15 | Not recorded | 6a0d405cf984739ed6737b93 | | TR | 2.05 | Not recorded | 6a0d405cf984739ed6737b93 | | TR | 1.87 | Not recorded | 6a0d405cf984739ed6737b93 | | Lookahead | 1.65 | Not recorded | 6a0d405cf984739ed6737b93 | | PLD | 1.55 | Not recorded | 6a0d405cf984739ed6737b93 | ### Graft MAT Score: 6.93. Result ID: 6a0d8303fbc373592c05fe4d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Vicuna-13B"}. This record does not assert independent reproduction. ### EAGLE3-2K MAT Score: 6.81. Result ID: 6a0d8303fbc373592c05fe4c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Vicuna-13B"}. This record does not assert independent reproduction. ### Graft MAT Score: 6.49. Result ID: 6a0d8303fbc373592c05fe51. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA3.1-8B"}. This record does not assert independent reproduction. ### EAGLE3-2K MAT Score: 6.42. Result ID: 6a0d8303fbc373592c05fe50. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA3.1-8B"}. This record does not assert independent reproduction. ### ECHO MAT Score: 6.35. Result ID: 6a0d8303fbc373592c05fe4b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Vicuna-13B"}. This record does not assert independent reproduction. ### ECHO MAT Score: 6.23. Result ID: 6a0d8303fbc373592c05fe4f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA3.1-8B"}. This record does not assert independent reproduction. ### DDD MAT Score: 6.07. Result ID: 6a0d8303fbc373592c05fe4a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Vicuna-13B"}. This record does not assert independent reproduction. ### Graft-DFLASH(16) MAT Score: 4.54. Result ID: 6a0d8303fbc373592c05fe99. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3-8B","Block Size":"16"}. This record does not assert independent reproduction. ### DFLASH(16) MAT Score: 4.39. Result ID: 6a0d8303fbc373592c05fe98. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Qwen3-8B","Block Size":"16"}. This record does not assert independent reproduction. ### Graft MAT Score: 3.79. Result ID: 6a0d8303fbc373592c05fe55. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-8B"}. This record does not assert independent reproduction. ### EAGLE3 MAT Score: 3.71. Result ID: 6a0d8303fbc373592c05fe54. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-8B"}. This record does not assert independent reproduction. ### ECHO MAT Score: 3.68. Result ID: 6a0d8303fbc373592c05fe53. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-8B"}. This record does not assert independent reproduction. ### Graft-EAGLE3(60) MAT Score: 3.41. Result ID: 6a0d8303fbc373592c05fe97. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3-8B","Budget":"60"}. This record does not assert independent reproduction. ### EAGLE3(60) MAT Score: 3.34. Result ID: 6a0d8303fbc373592c05fe96. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3-8B","Budget":"60"}. This record does not assert independent reproduction. ### Graft MAT Score: 3.15. Result ID: 6a0d8303fbc373592c05fe59. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-32B"}. This record does not assert independent reproduction. ### EAGLE3 MAT Score: 3.02. Result ID: 6a0d8303fbc373592c05fe58. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-32B"}. This record does not assert independent reproduction. ### ECHO MAT Score: 2.96. Result ID: 6a0d8303fbc373592c05fe57. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-32B"}. This record does not assert independent reproduction. ### Medusa MAT Score: 2.61. Result ID: 6a0d8303fbc373592c05fe46. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Vicuna-13B"}. This record does not assert independent reproduction. ### Graft MAT Score: 2.61. Result ID: 6a0d8303fbc373592c05fe5d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-235B"}. This record does not assert independent reproduction. ### EAGLE3-2K MAT Score: 2.56. Result ID: 6a0d8303fbc373592c05fe5c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-235B"}. This record does not assert independent reproduction. ### ECHO MAT Score: 2.38. Result ID: 6a0d8303fbc373592c05fe5b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-235B"}. This record does not assert independent reproduction. ### TR MAT Score: 2.37. Result ID: 6a0d8303fbc373592c05fe49. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Vicuna-13B"}. This record does not assert independent reproduction. ### TR MAT Score: 2.37. Result ID: 6a0d8303fbc373592c05fe56. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-32B"}. This record does not assert independent reproduction. ### TR MAT Score: 2.36. Result ID: 6a0d8303fbc373592c05fe52. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-8B"}. This record does not assert independent reproduction. ### Sps MAT Score: 2.26. Result ID: 6a0d8303fbc373592c05fe45. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Vicuna-13B"}. This record does not assert independent reproduction. ### SAMD MAT Score: 2.15. Result ID: 6a0d8303fbc373592c05fe48. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Vicuna-13B"}. This record does not assert independent reproduction. ### TR MAT Score: 2.05. Result ID: 6a0d8303fbc373592c05fe5a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-235B"}. This record does not assert independent reproduction. ### TR MAT Score: 1.87. Result ID: 6a0d8303fbc373592c05fe4e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA3.1-8B"}. This record does not assert independent reproduction. ### Lookahead MAT Score: 1.65. Result ID: 6a0d8303fbc373592c05fe44. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Vicuna-13B"}. This record does not assert independent reproduction. ### PLD MAT Score: 1.55. Result ID: 6a0d8303fbc373592c05fe47. [Reporting paper]() Recorded evaluation configuration: {"Model":"Vicuna-13B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.