LLM Judgment on Auto-J
56.68AccuracyGenii
Evaluation results
| Method | Links | |
|---|---|---|
| GeniiBackbone=Qwen2.5-14B-Instruct, Evaluation Setup=Same Model Family2025.10 | 56.68 | |
| VanillaBackbone=Qwen2.5-14B-Instruct, Evaluation Setup=Same Model Family2025.10 | 54.6 | |
| GeniiBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 53.38 | |
| GeniiBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 51.87 | |
| GeniiBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Same Model Family2025.10 | 51.8 | |
| Judgment SFTBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 51.15 | |
| Long ReasoningBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 50.43 | |
| Judgment SFTBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 50.43 | |
| Judgment SFTBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 48.99 | |
| Self-ConsistencyBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 48.88 | |
| VanillaBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 48.35 | |
| VanillaBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Same Model Family2025.10 | 48.35 | |
| Self-ConsistencyBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 47.88 | |
| VanillaBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 47.05 | |
| RepromptBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 46.41 | |
| Long ReasoningBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 45.4 | |
| RepromptBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 44.58 | |
| GeniiBackbone=Qwen2.5-3B-Instruct, Evaluation Setup=Same Model Family2025.10 | 42.03 | |
| VanillaBackbone=Qwen2.5-3B-Instruct, Evaluation Setup=Same Model Family2025.10 | 41.59 | |
| GeniiBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 40.73 | |
| Self-ConsistencyBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 39.35 | |
| VanillaBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 38.86 | |
| RepromptBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 27.3 |