List your company
ResearchFormerly WizWand

LLM-as-a-judge evaluation on FB Bench (Feedback Bench)

0.949Pearson's rQwen3-8B TRACT
0.166920.369960.5730.77604Mar 6, 2025May 7, 2025Jul 9, 2025Sep 10, 2025Nov 11, 2025Jan 13, 2026Mar 17, 2026
Updated 7mo ago

Evaluation results

MethodLinks
Qwen3-8B TRACTTraining=SFT, Inference=RAIL2026.030.9490.9470.843
Qwen3-8B Standard RLTraining=RL, Inference=RAIL2026.030.940.9420.835
Mistral2-7B TRACTTraining=SFT, Inference=RAIL2026.030.9390.9370.829
Mistral2-7B Standard RLTraining=RL, Inference=RAIL2026.030.9370.9370.828
Mistral-7B-Instruct (RAFT on GPT-4)Backbone=Mistral-7B-Instruct, CoT=false, Train Objective=RAFT†, Training Data=GPT-4†, Inference Method=RAIL2025.030.9320.93—
Mistral2-7B REAL (ours)Training=RL, Inference=RAIL2026.030.9320.9340.825
TRACTBackbone=Mistral-7B-Instruct, CoT=true, Train Objective=C-RAFT, Training Data=Self, Inference Method=C-RAIL2025.030.9310.93—
Baseline B.4 (RAFT)Backbone=Llama3.1-8B, COT=false, Train=RAFT, Data=GPT-4+, Inf=RAIL2025.030.920.918—
TRACTBackbone=Llama3.1-8B, COT=true, Train=C-RAFT, Data=Self, Inf=C-RAIL2025.030.920.917—
Qwen3-8B REAL (ours)Training=RL, Inference=RAIL2026.030.920.9210.857
TRACT (Ablation: Objective CE)Backbone=Mistral-7B-Instruct, CoT=true, Train Objective=CE, Training Data=Self, Inference Method=C-RAIL2025.030.9190.917—
Qwen3-32B REAL (ours)Training=RL, Inference=RAIL2026.030.9110.9170.859
Mistral-7B-Instruct (CE on GPT-4 scores)Backbone=Mistral-7B-Instruct, CoT=false, Train Objective=CE, Training Data=GPT-4†, Inference Method=Decode2025.030.890.891—
TRACT (Ablation: Data GPT-4)Backbone=Mistral-7B-Instruct, CoT=true, Train Objective=C-RAFT, Training Data=GPT-4, Inference Method=C-RAIL2025.030.8790.88—
Mistral2-7B RAFTTraining=SFT, Inference=RAIL2026.030.8790.880.763
Mistral-7B-Instruct (CE on Self-gen CoT)Backbone=Mistral-7B-Instruct, CoT=true, Train Objective=CE, Training Data=Self, Inference Method=Decode2025.030.8730.873—
Mistral-7B-Instruct (CE on GPT-4 CoT)Backbone=Mistral-7B-Instruct, CoT=true, Train Objective=CE, Training Data=GPT-4, Inference Method=Decode2025.030.8720.872—
Prometheus-1-13BTraining=SFT, Inference=Standard2026.030.860.8580.771
Baseline B.1 (CE)Backbone=Llama3.1-8B, COT=false, Train=CE, Data=GPT-4, Inf=Decode2025.030.8570.857—
Qwen3-32B RAFTTraining=SFT, Inference=RAIL2026.030.8540.8650.729
Prometheus-2-7BTraining=SFT, Inference=RAIL2026.030.8530.8530.729
Prometheus-1-7BTraining=SFT, Inference=Standard2026.030.8470.8490.767
Prometheus-2-7BBackbone=Prometheus-2-7B, Inference Method=Decode2025.030.8450.847—
Prometheus-2-7BTraining=SFT, Inference=Standard2026.030.8450.8470.765
Qwen3-8B RAFTTraining=SFT, Inference=RAIL2026.030.8430.8550.73
Mistral2-7B Base (w/ warmup)Training=None, Inference=RAIL2026.030.8370.8430.702
Baseline B.2 (CE)Backbone=Llama3.1-8B, COT=true, Train=CE, Data=GPT-4, Inf=Decode2025.030.8350.834—
Mistral2-7B Base (w/ warmup)Training=None, Inference=Standard2026.030.8310.8330.748
Baseline B.3 (RAIL)Backbone=Llama3.1-8B, COT=false, Inf=RAIL2025.030.6830.689—
TRACT (Ablation: Stage 1 Init)Backbone=Mistral-7B-Instruct, Note=Stage 2 initialized from ps, CoT=true2025.030.6740.684—
Qwen3-32B BaseTraining=None, Inference=RAIL2026.030.6340.7080.567
Qwen3-8B BaseTraining=None, Inference=RAIL2026.030.5670.6540.541
Qwen3-8B BaseTraining=None, Inference=Standard2026.030.5660.6270.539
GPT-3.5-Turbo-0613Training=None, Inference=Standard2026.030.5630.5210.453
CLoud (Reward model)Backbone=Llama-3-8B, Inference Method=Decode*2025.030.3810.376—
Mistral-7B-Instruct (Zero-shot RAIL)Backbone=Mistral-7B-Instruct, CoT=false, Inference Method=RAIL2025.030.1970.175—