LLM-as-a-Judge Robustness on Sage Easy
0.059Factuality Error (IPI)Gemini-2.5-Pro
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini-2.5-Pro2025.12 | 0.059 | 0.9 | 0.084 | 1.266 | 0.06 | 0.924 | 0.1 | 1.512 | 0.058 | 0.863 | 0.072 | 1.091 | |
| Qwen3-235B-A22B-Instruct-25072025.12 | 0.076 | 1.134 | 0.13 | 1.959 | 0.098 | 1.505 | 0.103 | 1.557 | 0.086 | 1.295 | 0.098 | 1.485 | |
| Gemini-2.5-Flash2025.12 | 0.077 | 1.163 | 0.111 | 1.679 | 0.075 | 1.175 | 0.097 | 1.504 | 0.071 | 1.081 | 0.087 | 1.326 | |
| DeepSeek-R1-05282025.12 | 0.095 | 1.436 | 0.142 | 2.183 | 0.141 | 2.254 | 0.124 | 1.857 | 0.087 | 1.3 | 0.115 | 1.759 | |
| Qwen3-4B-Instruct-25072025.12 | 0.101 | 1.532 | 0.175 | 2.662 | 0.126 | 1.933 | 0.124 | 1.943 | 0.102 | 1.532 | 0.126 | 1.919 | |
| Gemini-2.0-Flash-Lite2025.12 | 0.105 | 1.587 | 0.152 | 2.321 | 0.161 | 2.433 | 0.147 | 2.65 | 0.102 | 1.524 | 0.133 | 2.091 | |
| GPT-5-Chat2025.12 | 0.12 | 1.818 | 0.232 | 3.489 | 0.104 | 1.622 | 0.121 | 1.82 | 0.127 | 1.902 | 0.143 | 2.158 | |
| DeepSeek-V3-03242025.12 | 0.13 | 1.985 | 0.185 | 2.783 | 0.147 | 2.29 | 0.173 | 2.612 | 0.146 | 2.2 | 0.157 | 2.38 | |
| DeepSeek-V3.12025.12 | 0.134 | 2.022 | 0.189 | 2.92 | 0.146 | 2.449 | 0.204 | 3.097 | 0.132 | 2.017 | 0.16 | 2.485 | |
| Qwen3-30B-A3B-Instruct-25072025.12 | 0.136 | 2.056 | 0.208 | 3.113 | 0.123 | 1.9 | 0.171 | 2.569 | 0.173 | 2.597 | 0.162 | 2.448 | |
| GPT-4o-mini2025.12 | 0.15 | 2.294 | 0.202 | 3.085 | 0.164 | 2.522 | 0.107 | 1.667 | 0.182 | 2.733 | 0.164 | 2.496 | |
| Llama-3.1-8B-Instruct2025.12 | 0.204 | 3.203 | 0.229 | 3.506 | 0.26 | 4 | 0.23 | 3.493 | 0.179 | 2.755 | 0.221 | 3.41 | |
| Claude-3-Haiku2025.12 | 0.208 | 3.177 | 0.253 | 3.822 | 0.367 | 6.046 | 0.337 | 5.691 | 0.253 | 3.856 | 0.279 | 4.43 |