Structured Query Generation on KGQAGen-10k 2,000-question sample (train)
89.6LASM AccuracyAPEX-EM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| APEX-EMBackbone=Claude Sonnet 4.5, Config=Full memory + Opus judge (A5, E5)2026.03 | 89.6 | 95.3 | |
| APEX-EMBackbone=Claude Sonnet 4.5, Config=Full memory (EG2, E7)2026.03 | 88.6 | 95 | |
| APEX-EMBackbone=GPT4o, Config=Full memory + Opus judge (A5, E5)2026.03 | 87.8 | 89.2 | |
| APEX-EMBackbone=Claude Sonnet 4.5, Config=Semantic only (R1, E7)2026.03 | 85.7 | 94.8 | |
| APEX-EMBackbone=Claude Sonnet 4.5, Config=Entity graph only (EG1, E7)2026.03 | 85.5 | 94.8 | |
| GPT-4o (w/ SP)Category=LLM with Supporting Subgraph (oracle retrieval)2026.03 | 84.9 | — | |
| LLaMA2-7B (w/ SP)Category=LLM with Supporting Subgraph (oracle retrieval)2026.03 | 73.8 | — | |
| GPT-4.1Category=Pure LLMs (no retrieval)2026.03 | 58 | — | |
| GPT-4oCategory=Pure LLMs (no retrieval)2026.03 | 54.2 | — | |
| GCR (LLaMA-3.1 + GPT-4o)Category=KG-RAG Models, Fine-tuned=true2026.03 | 54 | — | |
| PoG (GPT-4o)Category=KG-RAG Models2026.03 | 53.5 | — | |
| ToG (GPT-4o)Category=KG-RAG Models2026.03 | 52.8 | — | |
| DeepSeek-ChatCategory=Pure LLMs (no retrieval)2026.03 | 50.1 | — | |
| GPT-4Category=Pure LLMs (no retrieval)2026.03 | 48.7 | — | |
| GPT-4o-miniCategory=Pure LLMs (no retrieval)2026.03 | 44.9 | — | |
| APEX-EMBackbone=Claude Sonnet 4.5, Config=No Memory (A0, 1 epoch)2026.03 | 41.3 | — | |
| LLaMA-3.1-8B-InstructCategory=Pure LLMs (no retrieval)2026.03 | 30.5 | — | |
| Mistral-7B-Instruct-v0.2Category=Pure LLMs (no retrieval)2026.03 | 22.8 | — | |
| RoG (LLaMA2-7B)Category=KG-RAG Models, Fine-tuned=true2026.03 | 21.3 | — | |
| LLaMA2-7BCategory=Pure LLMs (no retrieval)2026.03 | 11.6 | — |