LLM Judgment on UltraFeedback
68.75AccuracyGenii
Evaluation results
| Method | Links | |
|---|---|---|
| GeniiBackbone=Qwen2.5-14B-Instruct, Evaluation Setup=Same Model Family2025.10 | 68.75 | |
| VanillaBackbone=Qwen2.5-14B-Instruct, Evaluation Setup=Same Model Family2025.10 | 65.3 | |
| GeniiBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 64.75 | |
| GeniiBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 63.45 | |
| GeniiBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Same Model Family2025.10 | 63.15 | |
| VanillaBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 62.25 | |
| VanillaBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Same Model Family2025.10 | 62.25 | |
| Self-ConsistencyBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 62.15 | |
| Self-ConsistencyBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 61.54 | |
| VanillaBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 59.9 | |
| Judgment SFTBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 59.6 | |
| RepromptBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 58 | |
| Judgment SFTBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 57.25 | |
| Judgment SFTBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 56.7 | |
| GeniiBackbone=Qwen2.5-3B-Instruct, Evaluation Setup=Same Model Family2025.10 | 55.95 | |
| Long ReasoningBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 55.3 | |
| VanillaBackbone=Qwen2.5-3B-Instruct, Evaluation Setup=Same Model Family2025.10 | 55.3 | |
| Long ReasoningBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 54.65 | |
| RepromptBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 54.1 | |
| Self-ConsistencyBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 47.7 | |
| VanillaBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 47.2 | |
| GeniiBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 45.95 | |
| RepromptBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 35.1 |