Relevance on HotpotQA
0.9Kendall's TauJury-on-Demand
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Jury-on-Demand2025.12 | 0.9 | 0.02 | |
| GPT-OSS-120BJudge Model=GPT-OSS-120B2025.12 | 0.9 | 0.03 | |
| Gemini 2.5 FlashJudge Model=Gemini 2.5 Flash2025.12 | 0.89 | 0.03 | |
| Gemini 2.5 ProJudge Model=Gemini 2.5 Pro2025.12 | 0.89 | 0.03 | |
| GPT-OSS-20BJudge Model=GPT-OSS-20B2025.12 | 0.89 | 0.03 | |
| Static JuryAggregation Strategy=Average-TopK2025.12 | 0.87 | 0.03 | |
| Claude 3.7Judge Model=Claude 3.72025.12 | 0.87 | 0.03 | |
| Static JuryAggregation Strategy=Average-All2025.12 | 0.86 | 0.02 | |
| Static JuryAggregation Strategy=Weighted-Tau2025.12 | 0.86 | 0.03 | |
| Static JuryAggregation Strategy=Weighted-Regression2025.12 | 0.85 | 0.02 | |
| Gemini 2.0 FlashJudge Model=Gemini 2.0 Flash2025.12 | 0.85 | 0.03 | |
| Gemma 3Judge Model=Gemma 32025.12 | 0.78 | 0.03 | |
| DeepSeek R1Judge Model=DeepSeek R12025.12 | 0.6 | 0.05 | |
| Phi 4Judge Model=Phi 42025.12 | 0.45 | 0.05 | |
| LLAMA 3.2Judge Model=LL-3.22025.12 | 0.4 | 0.04 |