LLM-as-a-judge evaluation research benchmarks

Benchmarks

Dataset NameSOTA methodMetricTrendResultsLast Updated

MT-Bench

Qwen3-32B REAL (ours)

0.689Pearson's r

Mar 19, 2026

FLASK

Qwen3-32B REAL (ours)

0.589Pearson's r

Mar 19, 2026

FB Bench (Feedback Bench)

Qwen3-8B TRACT

0.949Pearson's r

Mar 19, 2026

JudgeBench (test)

Skywork-Reward-V2-Llama-3.1-8B-40M

83.4Score

Mar 4, 2026

Average Across FB Bench, FLASK, Vic. Bench, MT Bench

Qwen3-32B REAL (ours)

71Pearson (r)

Mar 19, 2026

Vicuna Benchmark

Qwen3-32B REAL (ours)

65.1Pearson Correlation (r)

Mar 19, 2026

Vicuna Bench

TRACT

0.605Pearson Correlation (r)

Feb 26, 2026

English (test)

Bucket-SFT

72.7Overall Score

Jun 12, 2026

Chinese (test)

Bucket-SFT

82.7Overall Score

Jun 12, 2026

100 Romanian synthetic prompts (test)

DeepSeek (Instruct)

4.71Fluency

Feb 26, 2026

Coding Qwen (n=8884)

AURA

77.74Adjusted Accuracy

Jun 19, 2026

Coding GPT-5.4

AURA

67.2Adjusted Accuracy

Jun 19, 2026