Citation Prediction Task 2 on CiteRAG 1.0 (test)
27.3PACA@10CitationGenerator-30B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CitationGenerator-30BSFT=Yes, RAG=R=102026.01 | 27.3 | 30.3 | 34.2 | |
| CitationGenerator-30BSFT=Yes, RAG=R=52026.01 | 26.8 | 29.1 | 31.8 | |
| CitationGenerator-30BSFT=Yes, RAG=None2026.01 | 14.9 | 16.5 | 19.1 | |
| Gemini-2.5-ProRAG=R=102026.01 | 12.7 | 17 | 20.1 | |
| CitationGenerator-4BSFT=Yes, RAG=R=102026.01 | 11.9 | 14.8 | 16.4 | |
| Qwen3-Coder-480BRAG=R=102026.01 | 11.3 | 14 | 16.1 | |
| Deepseek-v3RAG=R=102026.01 | 10.8 | 13.5 | 15.8 | |
| Gemini-2.0-FlashRAG=R=102026.01 | 10.7 | 13.8 | 17.1 | |
| Deepseek-v3RAG=R=52026.01 | 10.3 | 13.1 | 15.3 | |
| Grok-3RAG=R=102026.01 | 10.1 | 12.2 | 13.3 | |
| Qwen3-Coder-480BRAG=R=52026.01 | 10.1 | 12.1 | 13.8 | |
| Gemini-2.5-ProRAG=R=52026.01 | 9.9 | 12.9 | 15.5 | |
| GPT-5RAG=R=102026.01 | 9.8 | 12.1 | 14.3 | |
| Kimi-K2RAG=R=102026.01 | 9.8 | 12.3 | 14.3 | |
| CitationGenerator-30BSFT=No, RAG=R=102026.01 | 9.6 | 11.4 | 12.4 | |
| GPT-oss-120BRAG=R=102026.01 | 8.7 | 10.8 | 12.2 | |
| CitationGenerator-4BSFT=Yes, RAG=R=52026.01 | 8.7 | 9.6 | 10.1 | |
| GPT-5RAG=R=52026.01 | 8.6 | 10.9 | 13.1 | |
| Kimi-K2RAG=R=52026.01 | 8.5 | 10.4 | 12 | |
| Claude-3.7RAG=R=102026.01 | 8.2 | 10.5 | 12.5 | |
| Gemini-2.0-FlashRAG=R=52026.01 | 8.2 | 10.5 | 13.4 | |
| Grok-3RAG=R=52026.01 | 8.2 | 9.2 | 9.8 | |
| Claude-4-SonnetRAG=R=102026.01 | 8 | 10.1 | 12.2 | |
| Claude-4-SonnetRAG=R=52026.01 | 7.6 | 9.7 | 11.5 | |
| CitationGenerator-30BSFT=No, RAG=R=52026.01 | 7.6 | 8.4 | 8.9 | |
| Gemini-2.5-ProRAG=None2026.01 | 7.1 | 9.2 | 11.3 | |
| GPT-oss-120BRAG=R=52026.01 | 6.9 | 8.2 | 9 | |
| Claude-3.7RAG=R=52026.01 | 6.3 | 8.1 | 9.9 | |
| CitationGenerator-4BSFT=No, RAG=R=102026.01 | 6.3 | 7.8 | 9.2 | |
| Claude-3.5RAG=R=102026.01 | 5.7 | 6.9 | 7.8 | |
| GPT-5RAG=None2026.01 | 5.3 | 6.9 | 8.1 | |
| CitationGenerator-4BSFT=No, RAG=R=52026.01 | 5.1 | 6.4 | 7.4 | |
| Kimi-K2RAG=None2026.01 | 4.8 | 6.6 | 8 | |
| Deepseek-v3RAG=None2026.01 | 4.8 | 6.3 | 7.3 | |
| Claude-3.5RAG=R=52026.01 | 4.6 | 5.6 | 6.4 | |
| Claude-4-SonnetRAG=None2026.01 | 4.6 | 6.4 | 8 | |
| GPT-o3RAG=R=102026.01 | 4.5 | 5.8 | 6.8 | |
| Qwen3-Coder-480BRAG=None2026.01 | 4.2 | 5.3 | 6.5 | |
| Claude-3.7RAG=None2026.01 | 4 | 5.4 | 7.1 | |
| GPT-o3RAG=R=52026.01 | 3.9 | 4.9 | 6 | |
| GPT-o3RAG=None2026.01 | 3 | 3.6 | 4.1 | |
| Grok-3RAG=None2026.01 | 3 | 4.3 | 5.4 | |
| Gemini-2.0-FlashRAG=None2026.01 | 2.8 | 3.9 | 5.1 | |
| Claude-3.5RAG=None2026.01 | 2.4 | 2.9 | 3.4 | |
| GPT-oss-120BRAG=None2026.01 | 2.4 | 2.8 | 3.1 | |
| CitationGenerator-4BSFT=Yes, RAG=None2026.01 | 1.7 | 2.9 | 4.2 | |
| CitationGenerator-30BSFT=No, RAG=None2026.01 | 1.4 | 1.6 | 1.7 | |
| CitationGenerator-4BSFT=No, RAG=None2026.01 | 0.9 | 1 | 1.2 |