List your company
ResearchFormerly WizWand

Human Preference Ranking on Human Evaluation Elo (test)

1,634Elo Score(ai, ai+1)
909.121,097.311,285.51,473.69Sep 25, 2024Dec 10, 2024Feb 25, 2025May 13, 2025Jul 28, 2025Oct 13, 2025Dec 29, 2025
Updated 8mo ago

Evaluation results

MethodLinks
(ai, ai+1)training_pairs=1303, strategy=Stepwise preference pairs2025.121,634—
(ai, ai+1)dstraining_pairs=277, strategy=Stepwise preference pairs (downsampled)2025.121,620—
(a0, a*)training_pairs=277, strategy=Cumulative rewriting preference2025.121,612—
(a0, a1)training_pairs=277, strategy=Single-step edit preference2025.121,525—
(a, b)training_pairs=277, strategy=Standard A/B preference ranking2025.121,465—
basetraining_pairs=02025.121,383—
a* SFTtraining_pairs=277, strategy=Supervised Fine-Tuning2025.121,377—
GPT-4o-05132024.091,0791
Molmo-72BNumber of crops=122024.091,0772
Gemini 1.5 Pro2024.091,0743
Claude-3.5 Sonnet2024.091,0694
Llama-3.2V-90B-Instruct2024.091,0635
Molmo-7B-DNumber of crops=122024.091,0566
Gemini 1.5 Flash2024.091,0547
LLaVA OneVision-72B2024.091,0518
Molmo-7B-ONumber of crops=122024.091,0519
GPT-4V2024.091,04110
Llama-3.2V-11B-Instruct2024.091,04011
Qwen2-VL-72B2024.091,03712
MolmoE-1BNumber of crops=122024.091,03213
Qwen2-VL-7B2024.091,02514
LLaVA OneVision-7B2024.091,02415
InternVL2-Llama-3-76B2024.091,01816
Pixtral-12B2024.091,01617
Claude-3 Haiku2024.0999918
Phi3.5-Vision-4B2024.0998219
xGen-MM-interleave-4B2024.0997920
Claude-3 Opus2024.0997121
LLaVA-1.5-13B2024.0996022
InternVL2-8B2024.0995323
Cambrian-1-34B2024.0995324
Cambrian-1-8B2024.0995225
LLaVA-1.5-7B2024.0995126
PaliGemma-mix-3B2024.0993727