LLM Judgment on HotpotQA
67.24HSPPSelf-Consistency
Evaluation results
| Method | Links | |
|---|---|---|
| Self-ConsistencyBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 67.24 | |
| RepromptBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 63.58 | |
| VanillaBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 62.93 | |
| VanillaBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Same Model Family2025.10 | 62.93 | |
| Long ReasoningBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 60.96 | |
| Judgment SFTBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 53.23 | |
| GeniiBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 50.43 | |
| VanillaBackbone=Qwen2.5-14B-Instruct, Evaluation Setup=Same Model Family2025.10 | 45.47 | |
| Self-ConsistencyBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 44.51 | |
| VanillaBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 40.61 | |
| VanillaBackbone=Qwen2.5-3B-Instruct, Evaluation Setup=Same Model Family2025.10 | 40.52 | |
| RepromptBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 39.7 | |
| Judgment SFTBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 39.16 | |
| GeniiBackbone=Qwen2.5-14B-Instruct, Evaluation Setup=Same Model Family2025.10 | 39.01 | |
| RepromptBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 37.89 | |
| GeniiBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Same Model Family2025.10 | 34.7 | |
| GeniiBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 34.39 | |
| GeniiBackbone=Qwen2.5-3B-Instruct, Evaluation Setup=Same Model Family2025.10 | 21.12 | |
| Long ReasoningBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 3.28 | |
| VanillaBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 3.09 | |
| Judgment SFTBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 3.09 | |
| Self-ConsistencyBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 2.84 | |
| GeniiBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 1.55 |