Factual Retention on MINE
96.3Factual Retention (%)SoKG (Ours)
Evaluation Results
| Method | Links | |
|---|---|---|
| SoKG (Ours)LLM Backbone=Claude-4-Sonnet2026.01 | 96.3 | |
| SoKG (w/o 5W1H)LLM Backbone=Claude-4-Sonnet2026.01 | 94.6 | |
| SoKG (Ours)LLM Backbone=GPT-4o2026.01 | 89.3 | |
| SoKG (Ours)LLM Backbone=Gemini-2.5-Flash-Lite2026.01 | 87.7 | |
| Direct ExtractionLLM Backbone=Claude-4-Sonnet2026.01 | 86.8 | |
| SoKG (w/o 5W1H)LLM Backbone=Gemini-2.5-Flash-Lite2026.01 | 85.6 | |
| Direct ExtractionLLM Backbone=Gemini-2.5-Flash-Lite2026.01 | 84.6 | |
| SoKG (Ours)LLM Backbone=GPT-4o-mini2026.01 | 83.9 | |
| SoKG (w/o 5W1H)LLM Backbone=GPT-4o2026.01 | 83.5 | |
| SoKG (w/o 5W1H)LLM Backbone=GPT-4o-mini2026.01 | 80.5 | |
| Direct ExtractionLLM Backbone=GPT-4o2026.01 | 78.1 | |
| SoKG (Ours)LLM Backbone=Qwen2.5-7B-Instruct2026.01 | 73.4 | |
| KGGenLLM Backbone=Claude-4-Sonnet2026.01 | 69.1 | |
| Direct ExtractionLLM Backbone=GPT-4o-mini2026.01 | 68.5 | |
| SoKG (w/o 5W1H)LLM Backbone=Qwen2.5-7B-Instruct2026.01 | 67.1 | |
| Direct ExtractionLLM Backbone=Qwen2.5-7B-Instruct2026.01 | 66.5 | |
| KGGenLLM Backbone=GPT-4o2026.01 | 66.4 | |
| KGGenLLM Backbone=Gemini-2.5-Flash-Lite2026.01 | 62.5 | |
| GraphRAGLLM Backbone=Qwen2.5-7B-Instruct2026.01 | 59.7 | |
| KGGenLLM Backbone=Qwen2.5-7B-Instruct2026.01 | 56.7 | |
| GraphRAGLLM Backbone=Claude-4-Sonnet2026.01 | 52.3 | |
| GraphRAGLLM Backbone=GPT-4o-mini2026.01 | 49.5 | |
| GraphRAGLLM Backbone=GPT-4o2026.01 | 49.3 | |
| GraphRAGLLM Backbone=Gemini-2.5-Flash-Lite2026.01 | 48.5 | |
| KGGenLLM Backbone=GPT-4o-mini2026.01 | 44.3 |