Temporal Knowledge Graph Question Answering on MultiTQ (test)
77.9Hits@1 (Overall)MemoTime
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MemoTimeLLM=GPT-4-Turbo2025.10 | 77.9 | 53.8 | 86.8 | 74.5 | 85.3 | |
| MemoTimeLLM=DeepSeek-V32025.10 | 73 | 45.9 | 82.9 | 67.7 | 84.6 | |
| MemoTimeLLM=Qwen3-32B2025.10 | 68.2 | 42.5 | 77.6 | 62.1 | 81.7 | |
| MemoTimeLLM=GPT-4o-mini2025.10 | 64.2 | 40.3 | 73 | 61.5 | 70.8 | |
| TempAgentLLM=GPT-3.5-Turbo2025.10 | 53.9 | 16.8 | 68.4 | 47.8 | 66.1 | |
| ARILLM=GPT-3.5-Turbo2025.10 | 38 | 68 | 21 | 39.4 | 34.4 | |
| MultiQALLM=None2025.10 | 29.3 | 15.9 | 34.7 | 34.9 | 15.7 | |
| CronKGQALLM=None2025.10 | 27.9 | 13.4 | 33.7 | 32.8 | 15.6 | |
| ReAct KBLLM=GPT-3.5-Turbo2025.10 | 21 | 13.6 | 63.5 | 31.3 | 30 | |
| EmbedKGQALLM=None2025.10 | 20.6 | 13.4 | 23.5 | 29 | 0.1 | |
| KG-RAGLLM=GPT-3.5-Turbo2025.10 | 18.5 | 16 | 20 | 23 | 7 | |
| ALBERTLLM=None2025.10 | 10.8 | 8.6 | 11.6 | 13.9 | 3.2 | |
| ChatGPTLLM=GPT-3.5-Turbo2025.10 | 10.2 | 7.7 | 14.7 | 13.7 | 2 | |
| BERTLLM=None2025.10 | 8.3 | 6.1 | 9.2 | 10.1 | 4 | |
| AT2QACategory=Autonomous Training-free Agent, Training-mode=training-free2026.03 | 0.887 | 0.751 | 0.942 | 0.864 | 0.945 | |
| Temp-R1Category=LLM-based Static Workflows, Note=extensively fine-tuned2026.03 | 0.78 | 0.55 | 0.888 | 0.714 | 0.969 | |
| PoKCategory=LLM-based Static Workflows2026.03 | 0.779 | 0.409 | 0.929 | 0.696 | 0.962 | |
| RTQACategory=LLM-based Static Workflows2026.03 | 0.765 | 0.424 | 0.902 | 0.692 | 0.942 | |
| MemoTimeCategory=LLM-based Static Workflows2026.03 | 0.73 | 0.459 | 0.829 | 0.677 | 0.846 | |
| TimeRCategory=LLM-based Static Workflows2026.03 | 0.728 | 0.335 | 0.887 | 0.639 | 0.945 | |
| TempAgentCategory=LLM-based Static Workflows2026.03 | 0.702 | 0.316 | 0.857 | 0.624 | 0.87 | |
| ARICategory=LLM-based Static Workflows2026.03 | 0.38 | 0.21 | 0.68 | 0.394 | 0.344 | |
| Search-R1Category=LLM-based Static Workflows2026.03 | 0.352 | 0.094 | 0.474 | 0.23 | 0.705 | |
| MultiQACategory=TKG Embedding-based Methods2026.03 | 0.293 | 0.159 | 0.347 | 0.349 | 0.157 | |
| CronKGQACategory=TKG Embedding-based Methods2026.03 | 0.279 | 0.134 | 0.337 | 0.328 | 0.156 | |
| EmbedKGQACategory=TKG Embedding-based Methods2026.03 | 0.206 | 0.134 | 0.235 | 0.29 | 0.001 |