LLM Judgment on Natural Questions (NQ)
65.2HSPPSelf-Consistency
Evaluation results
| Method | Links | |
|---|---|---|
| Self-ConsistencyBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 65.2 | |
| VanillaBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 63.49 | |
| VanillaBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Same Model Family2025.10 | 63.49 | |
| Long ReasoningBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 62.3 | |
| RepromptBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 61.22 | |
| Judgment SFTBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 58.66 | |
| GeniiBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 52.13 | |
| VanillaBackbone=Qwen2.5-14B-Instruct, Evaluation Setup=Same Model Family2025.10 | 48.15 | |
| RepromptBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 40.51 | |
| RepromptBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 39.93 | |
| Self-ConsistencyBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 38.85 | |
| GeniiBackbone=Qwen2.5-14B-Instruct, Evaluation Setup=Same Model Family2025.10 | 38.07 | |
| VanillaBackbone=Qwen2.5-3B-Instruct, Evaluation Setup=Same Model Family2025.10 | 35.08 | |
| VanillaBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 32.37 | |
| Judgment SFTBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 30.1 | |
| GeniiBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Same Model Family2025.10 | 29.55 | |
| GeniiBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 28.9 | |
| GeniiBackbone=Qwen2.5-3B-Instruct, Evaluation Setup=Same Model Family2025.10 | 17.33 | |
| VanillaBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 6.96 | |
| Self-ConsistencyBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 6.33 | |
| Judgment SFTBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 5.7 | |
| GeniiBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 5.06 | |
| Long ReasoningBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 1 |