# Multi-Category Quality Scoring on MT-Bench

Source: [Multi-Category Quality Scoring on MT-Bench](<https://www.sota2.com/research/sota/multi-category-quality-scoring-on-mt-bench>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-28T09:55:27.629Z

Evaluation of LLM performance on the MT-Bench dataset, reporting scores across eight distinct categories (Writing, Reasoning, Coding, etc.) and a single overall score, typically derived from LLM-as-a-Judge assessments.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | General LLM Evaluation |
| Dataset (including any stated split) | MT-Bench |
| Metric | Overall Score |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 7 of 7 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Overall Score](<https://www.sota2.com/research/sota/multi-category-quality-scoring-on-mt-bench.md?metric=Overall%20Score>): Higher is better; unit: SCALAR.

- [Writing](<https://www.sota2.com/research/sota/multi-category-quality-scoring-on-mt-bench.md?metric=Writing>): Higher is better; unit: SCALAR.

- [Roleplay](<https://www.sota2.com/research/sota/multi-category-quality-scoring-on-mt-bench.md?metric=Roleplay>): Higher is better; unit: SCALAR.

- [Reasoning](<https://www.sota2.com/research/sota/multi-category-quality-scoring-on-mt-bench.md?metric=Reasoning>): Higher is better; unit: SCALAR.

- [Math](<https://www.sota2.com/research/sota/multi-category-quality-scoring-on-mt-bench.md?metric=Math>): Higher is better; unit: SCALAR.

- [Coding](<https://www.sota2.com/research/sota/multi-category-quality-scoring-on-mt-bench.md?metric=Coding>): Higher is better; unit: SCALAR.

- [Extraction](<https://www.sota2.com/research/sota/multi-category-quality-scoring-on-mt-bench.md?metric=Extraction>): Higher is better; unit: SCALAR.

- [STEM](<https://www.sota2.com/research/sota/multi-category-quality-scoring-on-mt-bench.md?metric=STEM>): Higher is better; unit: SCALAR.

- [Humanities](<https://www.sota2.com/research/sota/multi-category-quality-scoring-on-mt-bench.md?metric=Humanities>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Overall Score | Result date | Report paper |
| --- | --- | --- | --- |
| AdaDPO | 8.03 | Not recorded | 6a17cc4bdbd7150506f8153d |
| DPO | 8.01 | Not recorded | 6a17cc4bdbd7150506f8153d |
| SFT | 7.58 | Not recorded | 6a17cc4bdbd7150506f8153d |
| 16-bit Baseline | 6.38 | Not recorded | 68fa78f21e209131cea7b600 |
| AMXFP4 | 5.73 | Not recorded | 68fa78f21e209131cea7b600 |
| MXFP4 | 5.25 | Not recorded | 68fa78f21e209131cea7b600 |
| MXFP4-PoT | 3.22 | Not recorded | 68fa78f21e209131cea7b600 |

### AdaDPO

Overall Score: 8.03. Result ID: 6a18110afbc373592c124e75.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc4bdbd7150506f8153d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a17cc4bdbd7150506f8153d.md>)

Recorded evaluation configuration: {"Configuration Selection":"Best across 16 (lr, beta) combinations"}. This record does not assert independent reproduction.

### DPO

Overall Score: 8.01. Result ID: 6a18110afbc373592c124e74.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc4bdbd7150506f8153d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74511e209131cea6105d.md>)

Recorded evaluation configuration: {"Configuration Selection":"Best across 16 (lr, beta) combinations"}. This record does not assert independent reproduction.

### SFT

Overall Score: 7.58. Result ID: 6a18110afbc373592c124e73.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc4bdbd7150506f8153d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75d11e209131cea6a028.md>)

[Implementation](<https://github.com/meta-llama/llama-agentic-system>)

Recorded evaluation configuration: {"Configuration Selection":"Best across 16 (lr, beta) combinations"}. This record does not assert independent reproduction.

### 16-bit Baseline

Overall Score: 6.38. Result ID: 6995b2293badfaffe6b1f150.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78f21e209131cea7b600.md>)

Recorded evaluation configuration: {"Data Format":"16-bit Baseline","Model":"LLaMA2-Chat-7B"}. This record does not assert independent reproduction.

### AMXFP4

Overall Score: 5.73. Result ID: 6995b2293badfaffe6b1f153.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78f21e209131cea7b600.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78f21e209131cea7b600.md>)

[Implementation](<https://www.sota2.com/aiha-lab/MX-QLLM.git>)

Recorded evaluation configuration: {"Data Format":"AMXFP4","Model":"LLaMA2-Chat-7B"}. This record does not assert independent reproduction.

### MXFP4

Overall Score: 5.25. Result ID: 6995b2293badfaffe6b1f152.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78f21e209131cea7b600.md>)

Recorded evaluation configuration: {"Data Format":"MXFP4","Model":"LLaMA2-Chat-7B"}. This record does not assert independent reproduction.

### MXFP4-PoT

Overall Score: 3.22. Result ID: 6995b2293badfaffe6b1f151.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78f21e209131cea7b600.md>)

Recorded evaluation configuration: {"Data Format":"MXFP4-PoT","Model":"LLaMA2-Chat-7B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
