Temporal Question Answering on TimeQA Easy
63.4R-1o4-mini
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| o4-miniPrompt Type=W/ ORIGINAL CONTEXT (C), Model Type=Close-Source2026.02 | 63.4 | 51.9 | 63.2 | 71.1 | 4,414 | — | |
| GPT-4oPrompt Type=W/ TIME SUB-C, Model Type=Close-Source2026.02 | 62.2 | 51 | 62.1 | 71.9 | 4,515 | — | |
| GPT-4oPrompt Type=W/ ORIGINAL CONTEXT (C), Model Type=Close-Source2026.02 | 56 | 45.6 | 55.9 | 63.5 | 3,995 | — | |
| GPT-3.5Prompt Type=W/ TIME SUB-C, Model Type=Close-Source2026.02 | 46.1 | 38.2 | 46 | 60.9 | 3,144 | — | |
| GPT-3.5Prompt Type=W/ ORIGINAL CONTEXT (C), Model Type=Close-Source2026.02 | 44.3 | 36.4 | 44.2 | 60.8 | 3,018 | — | |
| GPT-4oPrompt Type=W/ KGS (WHOLE-C) (FAISS), Model Type=Close-Source2026.02 | 40.9 | 34.4 | 40.8 | 55.9 | 3,010 | — | |
| GPT-4oPrompt Type=W/ KGS (CHUNKED-C) (KEYBERT), Model Type=Close-Source2026.02 | 33.8 | 28.8 | 33.8 | 50.5 | 2,672 | — | |
| Qwen3-4B-ThinkPrompt Type=W/ ORIGINAL CONTEXT (C), Model Type=Open-Source2026.02 | 32.8 | 27.1 | 27.1 | 37.1 | 2,456 | — | |
| GPT-4oPrompt Type=W/ KGS (CHUNKED-C) (FAISS), Model Type=Close-Source2026.02 | 32.8 | 27.9 | 32.7 | 49.6 | 2,466 | — | |
| GPT-4oPrompt Type=ONLY QUESTION, Model Type=Close-Source2026.02 | 31 | 25.3 | 31 | 49 | 2,156 | — | |
| GPT-3.5Prompt Type=W/ KGS (WHOLE-C) (FAISS), Model Type=Close-Source2026.02 | 28.3 | 24.7 | 28.3 | 46.7 | 2,012 | — | |
| GPT-3.5Prompt Type=W/ KGS (CHUNKED-C) (KEYBERT), Model Type=Close-Source2026.02 | 23.1 | 20 | 23 | 42.5 | 1,664 | — | |
| GPT-3.5Prompt Type=W/ KGS (CHUNKED-C) (FAISS), Model Type=Close-Source2026.02 | 22 | 19.7 | 22 | 41.6 | 1,505 | — | |
| Qwen2.5-7BPrompt Type=W/ ORIGINAL CONTEXT (C), Model Type=Open-Source2026.02 | 18.1 | 11.4 | 18.1 | 43.9 | 890 | — | |
| GPT-3.5Prompt Type=ONLY QUESTION, Model Type=Close-Source2026.02 | 17.2 | 14.7 | 17.2 | 37.7 | 740 | — | |
| Llama3.2-3BPrompt Type=ONLY QUESTION, Model Type=Open-Source2026.02 | 12.9 | 8 | 12.9 | 39.8 | 255 | — | |
| Qwen2.5-1.5BPrompt Type=W/ ORIGINAL CONTEXT (C), Model Type=Open-Source2026.02 | 12.7 | 9.6 | 12.7 | 38 | 806 | — | |
| Qwen2.5-7BPrompt Type=ONLY QUESTION, Model Type=Open-Source2026.02 | 11.6 | 9.1 | 11.6 | 37.8 | 816 | — | |
| Qwen2.5-1.5BPrompt Type=ONLY QUESTION, Model Type=Open-Source2026.02 | 11.5 | 11.1 | 11.5 | 33.2 | 1,113 | — | |
| Qwen2.5-0.5BPrompt Type=ONLY QUESTION, Model Type=Open-Source2026.02 | 6.2 | 5.2 | 6.2 | 30.9 | 0 | — | |
| DYNA2026.06 | — | — | — | — | — | 75.67 | |
| DynaGen2026.06 | — | — | — | — | — | 73.28 | |
| RAG2026.06 | — | — | — | — | — | 68.23 |