# Pairwise LLM Judging on MT-Bench Source: [Pairwise LLM Judging on MT-Bench]() Collection index: [llms.txt]() Data updated: 2026-06-01T07:55:00.294Z Evaluation of LLM performance on the MT-Bench dataset using pairwise comparison judging, focusing on metrics such as Coverage and Risk. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Pairwise LLM Judging | | Dataset (including any stated split) | MT-Bench | | Metric | Coverage | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 25 of 25 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Coverage](): Higher is better; unit: PERCENT. - [Risk](): Lower is better; unit: SCALAR. - [Human Agreement](): Higher is better; unit: PERCENT. ## Recorded results | Method | Coverage | Result date | Report paper | | --- | --- | --- | --- | | Vanilla | 100 | Not recorded | 6992a4bc9ebc499c97406dba | | Vanilla | 100 | Not recorded | 6992a4bc9ebc499c97406dba | | Vanilla | 100 | Not recorded | 6992a4bc9ebc499c97406dba | | Vanilla | 100 | Not recorded | 6992a4bc9ebc499c97406dba | | Heuristic | 95.3 | Not recorded | 6992a4bc9ebc499c97406dba | | Heuristic | 93.3 | Not recorded | 6992a4bc9ebc499c97406dba | | Heuristic | 90.7 | Not recorded | 6992a4bc9ebc499c97406dba | | Heuristic | 80.9 | Not recorded | 6992a4bc9ebc499c97406dba | | Naive | 56.6 | Not recorded | 6992a4bc9ebc499c97406dba | | SCOPE | 54.3 | Not recorded | 6992a4bc9ebc499c97406dba | | SCOPE | 50.4 | Not recorded | 6992a4bc9ebc499c97406dba | | SCOPE | 50.2 | Not recorded | 6992a4bc9ebc499c97406dba | | Naive | 42.6 | Not recorded | 6992a4bc9ebc499c97406dba | | Naive | 39.6 | Not recorded | 6992a4bc9ebc499c97406dba | | SCOPE | 24.6 | Not recorded | 6992a4bc9ebc499c97406dba | | Naive | 10.2 | Not recorded | 6992a4bc9ebc499c97406dba | ### Vanilla Coverage: 100. Result ID: 699a62665698665ec4ea72ea. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-7B","alpha":"0.10"}. This record does not assert independent reproduction. ### Vanilla Coverage: 100. Result ID: 699a62665698665ec4ea72ee. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-14B","alpha":"0.10"}. This record does not assert independent reproduction. ### Vanilla Coverage: 100. Result ID: 699a62665698665ec4ea72f2. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-32B","alpha":"0.10"}. This record does not assert independent reproduction. ### Vanilla Coverage: 100. Result ID: 699a62665698665ec4ea72f6. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Llama-70B","alpha":"0.10"}. This record does not assert independent reproduction. ### Heuristic Coverage: 95.3. Result ID: 699a62665698665ec4ea72ef. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-14B","alpha":"0.10"}. This record does not assert independent reproduction. ### Heuristic Coverage: 93.3. Result ID: 699a62665698665ec4ea72f3. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-32B","alpha":"0.10"}. This record does not assert independent reproduction. ### Heuristic Coverage: 90.7. Result ID: 699a62665698665ec4ea72eb. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-7B","alpha":"0.10"}. This record does not assert independent reproduction. ### Heuristic Coverage: 80.9. Result ID: 699a62665698665ec4ea72f7. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Llama-70B","alpha":"0.10"}. This record does not assert independent reproduction. ### Naive Coverage: 56.6. Result ID: 699a62665698665ec4ea72f0. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-14B","alpha":"0.10"}. This record does not assert independent reproduction. ### SCOPE Coverage: 54.3. Result ID: 699a62665698665ec4ea72f9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Judge Model":"Llama-70B","alpha":"0.10"}. This record does not assert independent reproduction. ### SCOPE Coverage: 50.4. Result ID: 699a62665698665ec4ea72f1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-14B","alpha":"0.10"}. This record does not assert independent reproduction. ### SCOPE Coverage: 50.2. Result ID: 699a62665698665ec4ea72f5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-32B","alpha":"0.10"}. This record does not assert independent reproduction. ### Naive Coverage: 42.6. Result ID: 699a62665698665ec4ea72f4. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-32B","alpha":"0.10"}. This record does not assert independent reproduction. ### Naive Coverage: 39.6. Result ID: 699a62665698665ec4ea72f8. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Llama-70B","alpha":"0.10"}. This record does not assert independent reproduction. ### SCOPE Coverage: 24.6. Result ID: 699a62665698665ec4ea72ed. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-7B","alpha":"0.10"}. This record does not assert independent reproduction. ### Naive Coverage: 10.2. Result ID: 699a62665698665ec4ea72ec. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-7B","alpha":"0.10"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.