Question Answering on SpecsQA (test)
29.8F1 (Factual Correctness)Router with TKG fallback
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Router with TKG fallback2026.05 | 29.8 | 35 | 31.2 | 35.7 | 49 | 34.4 | 0.64 | |
| SKG with TKG fallback2026.05 | 29.3 | 34.6 | 30.9 | 37.2 | 51.8 | 35 | 0.644 | |
| RAPTOR2026.05 | 20.7 | 30.1 | 20 | 21.6 | 37.3 | 20 | 0.568 | |
| Microsoft GraphRAG2026.05 | 15.3 | 24 | 15.2 | 20.3 | 37.5 | 18.8 | 0.575 | |
| HippoRAG 22026.05 | 14.9 | 26.7 | 16.6 | 15.2 | 33.6 | 12.5 | 0.54 | |
| LinearRAG2026.05 | 13.8 | 19.7 | 16.3 | 8.5 | 17.2 | 7.9 | 0.526 | |
| Wikontic2026.05 | 13.5 | 35.1 | 10.7 | 29 | 52.2 | 25.3 | 0.523 | |
| LLM only2026.05 | 10.7 | 13.6 | 11.1 | 2.8 | 4.6 | 2.6 | 0.559 | |
| Vector RAG2026.05 | 9.2 | 16.1 | 8.5 | 11.8 | 24.4 | 10.6 | 0.438 | |
| TableRAG2026.05 | 9.1 | 12.3 | 9.3 | 11.8 | 23.7 | 11 | 0.453 | |
| Microsoft GraphRAGmode=fast2026.05 | 8.3 | 13 | 8.2 | 14 | 20.5 | 13.9 | 0.42 | |
| AriGraph2026.05 | 7.2 | 15.5 | 6.9 | 12.4 | 19.9 | 11.9 | 0.378 | |
| A-RAG2026.05 | 5.7 | 4.7 | 10.2 | 12.9 | 16.4 | 12.4 | 0.331 |