Citation-grounded summarization on Citation-grounded summarization (test)
95.8Factual AccuracyReference Model
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Reference ModelAccess=Closed-source, Capabilities=Frontier model with extended reasoning2026.01 | 95.8 | 70.5 | 98.9 | 92.6 | 87.4 | 89.1 | |
| Gemma-3-27B-InstructParameters=27B2026.01 | 91.6 | 36.8 | 91.6 | 14.7 | 43.2 | 55.6 | |
| sui-1Precision=Standard2026.01 | 90.5 | 60 | 97.9 | 89.5 | 83.2 | 84.2 | |
| Qwen3-32B-InstructParameters=32B2026.01 | 90.5 | 22.1 | 91.6 | 17.9 | 37.9 | 52 | |
| Mistral-Small-3.2-24B-InstructParameters=24B2026.01 | 90.5 | 12.6 | 93.7 | 13.7 | 38.9 | 49.9 | |
| sui-1Precision=FP8 quantization2026.01 | 88.4 | 51.6 | 97.9 | 92.6 | 74.7 | 81.1 | |
| Llama-3.3-70B-InstructParameters=70B2026.01 | 81.1 | 4.2 | 67.4 | 41.1 | 20 | 42.7 |