# Pairwise LLM Judging on Chatbot Arena Source: [Pairwise LLM Judging on Chatbot Arena]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:40:04.065Z Evaluation of LLM performance using pairwise comparison judging on the Chatbot Arena dataset, focusing on metrics like Coverage and Risk. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Pairwise LLM Judging | | Dataset (including any stated split) | Chatbot Arena | | Metric | Coverage | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Coverage](): Higher is better; unit: PERCENT. - [Risk](): Lower is better; unit: SCALAR. ## Recorded results | Method | Coverage | Result date | Report paper | | --- | --- | --- | --- | | Vanilla | 100 | Not recorded | 6992a4bc9ebc499c97406dba | | Vanilla | 100 | Not recorded | 6992a4bc9ebc499c97406dba | | Vanilla | 100 | Not recorded | 6992a4bc9ebc499c97406dba | | Vanilla | 100 | Not recorded | 6992a4bc9ebc499c97406dba | | Heuristic | 95.8 | Not recorded | 6992a4bc9ebc499c97406dba | | Heuristic | 93.7 | Not recorded | 6992a4bc9ebc499c97406dba | | Heuristic | 90.3 | Not recorded | 6992a4bc9ebc499c97406dba | | Heuristic | 81.5 | Not recorded | 6992a4bc9ebc499c97406dba | | SCOPE | 58.3 | Not recorded | 6992a4bc9ebc499c97406dba | | Naive | 56.5 | Not recorded | 6992a4bc9ebc499c97406dba | | SCOPE | 55.1 | Not recorded | 6992a4bc9ebc499c97406dba | | Naive | 53.2 | Not recorded | 6992a4bc9ebc499c97406dba | | Naive | 53 | Not recorded | 6992a4bc9ebc499c97406dba | | SCOPE | 46.6 | Not recorded | 6992a4bc9ebc499c97406dba | | SCOPE | 43 | Not recorded | 6992a4bc9ebc499c97406dba | | Naive | 29.7 | Not recorded | 6992a4bc9ebc499c97406dba | ### Vanilla Coverage: 100. Result ID: 699a62665698665ec4ea730a. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-7B","alpha":"0.10"}. This record does not assert independent reproduction. ### Vanilla Coverage: 100. Result ID: 699a62665698665ec4ea730e. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-14B","alpha":"0.10"}. This record does not assert independent reproduction. ### Vanilla Coverage: 100. Result ID: 699a62665698665ec4ea7312. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-32B","alpha":"0.10"}. This record does not assert independent reproduction. ### Vanilla Coverage: 100. Result ID: 699a62665698665ec4ea7316. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Llama-70B","alpha":"0.10"}. This record does not assert independent reproduction. ### Heuristic Coverage: 95.8. Result ID: 699a62665698665ec4ea730f. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-14B","alpha":"0.10"}. This record does not assert independent reproduction. ### Heuristic Coverage: 93.7. Result ID: 699a62665698665ec4ea7313. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-32B","alpha":"0.10"}. This record does not assert independent reproduction. ### Heuristic Coverage: 90.3. Result ID: 699a62665698665ec4ea730b. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-7B","alpha":"0.10"}. This record does not assert independent reproduction. ### Heuristic Coverage: 81.5. Result ID: 699a62665698665ec4ea7317. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Llama-70B","alpha":"0.10"}. This record does not assert independent reproduction. ### SCOPE Coverage: 58.3. Result ID: 699a62665698665ec4ea7319. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Judge Model":"Llama-70B","alpha":"0.10"}. This record does not assert independent reproduction. ### Naive Coverage: 56.5. Result ID: 699a62665698665ec4ea7318. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Llama-70B","alpha":"0.10"}. This record does not assert independent reproduction. ### SCOPE Coverage: 55.1. Result ID: 699a62665698665ec4ea7315. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-32B","alpha":"0.10"}. This record does not assert independent reproduction. ### Naive Coverage: 53.2. Result ID: 699a62665698665ec4ea7314. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-32B","alpha":"0.10"}. This record does not assert independent reproduction. ### Naive Coverage: 53. Result ID: 699a62665698665ec4ea7310. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-14B","alpha":"0.10"}. This record does not assert independent reproduction. ### SCOPE Coverage: 46.6. Result ID: 699a62665698665ec4ea7311. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-14B","alpha":"0.10"}. This record does not assert independent reproduction. ### SCOPE Coverage: 43. Result ID: 699a62665698665ec4ea730d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-7B","alpha":"0.10"}. This record does not assert independent reproduction. ### Naive Coverage: 29.7. Result ID: 699a62665698665ec4ea730c. [Reporting paper]() Recorded evaluation configuration: {"Judge Model":"Qwen-7B","alpha":"0.10"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.