LLM-as-a-Judge Robustness on Sage (Hard)
55.9Factuality (IPI)Llama-3.1-8B-Instruct
Evaluation Results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Llama-3.1-8B-Instruct2025.12 | 55.9 | 8.507 | 49.6 | 7.6 | 65.3 | 10.254 | 58.1 | 9.246 | 60.8 | 9.265 | 57.3 | 8.869 | |
| GPT-4o-mini2025.12 | 54.6 | 8.261 | 47.8 | 7.276 | 45.2 | 7.803 | 35.3 | 5.755 | 66.1 | 9.974 | 50.2 | 7.865 | |
| Claude-3-Haiku2025.12 | 52.2 | 8.442 | 50.7 | 7.985 | 29.2 | 5.213 | 35.2 | 6.171 | 61 | 10.161 | 46.4 | 7.687 | |
| DeepSeek-V3.12025.12 | 51.3 | 8.63 | 58.5 | 9.171 | 21.4 | 3.893 | 43.8 | 7.235 | 52.3 | 9.47 | 46 | 7.756 | |
| DeepSeek-V3-03242025.12 | 45.5 | 7.146 | 45.5 | 7.06 | 49 | 8.286 | 40.3 | 6.331 | 53.3 | 8.492 | 46.7 | 7.446 | |
| Gemini-2.0-Flash-Lite2025.12 | 43.4 | 6.594 | 37.6 | 5.764 | 45.1 | 7.417 | 33.6 | 5.919 | 48 | 7.282 | 41.5 | 6.571 | |
| GPT-5-Chat2025.12 | 43.3 | 6.563 | 57.1 | 8.619 | 28.7 | 4.85 | 28.6 | 4.431 | 58.3 | 8.764 | 43.6 | 6.7 | |
| Qwen3-30B-A3B-Instruct-25072025.12 | 42.4 | 6.458 | 39.3 | 6.059 | 33.7 | 5.625 | 37.9 | 6.065 | 53 | 7.968 | 41.3 | 6.435 | |
| DeepSeek-R1-05282025.12 | 41 | 6.397 | 45.1 | 7.067 | 27.6 | 5.032 | 39.8 | 6.404 | 43.4 | 6.828 | 39.6 | 6.362 | |
| Qwen3-4B-Instruct-25072025.12 | 37.6 | 5.734 | 40.5 | 6.215 | 37.6 | 6.254 | 34.6 | 5.653 | 43.3 | 6.573 | 38.8 | 6.078 | |
| Qwen3-235B-A22B-Instruct-25072025.12 | 34.7 | 5.326 | 35.7 | 5.585 | 25.6 | 4.404 | 26.8 | 4.124 | 42 | 6.409 | 33.1 | 5.183 | |
| Gemini-2.5-Flash2025.12 | 26.5 | 4.175 | 32 | 5.007 | 25.7 | 4.457 | 22.8 | 3.732 | 26.7 | 4.331 | 26.9 | 4.35 | |
| Gemini-2.5-Pro2025.12 | 24 | 3.769 | 29.1 | 4.586 | 19.3 | 3.661 | 26.8 | 4.439 | 25 | 3.903 | 25 | 4.079 |