Temporal Question Answering on TransientTables (test)
80.39EMGemini-2.0-Flash
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Gemini-2.0-FlashSchema Generator=Gemini 2.5 Flash, Approach=Evidence-Guided Schema Normalization2025.11 | 80.39 | 82.11 | |
| Llama-3.3-70B-InstructSchema Generator=Llama-3.3-70B-Instruct, Approach=Evidence-Guided Schema Normalization2025.11 | 79.52 | 79.91 | |
| Llama-3.1-8B-InstructSchema Generator=Llama-3.3-70B-Instruct, Approach=Evidence-Guided Schema Normalization2025.11 | 78.7 | 77.85 | |
| GPT-4o-miniSchema Generator=Llama-3.3-70B-Instruct, Approach=Evidence-Guided Schema Normalization2025.11 | 78.54 | 78.7 | |
| Gemini-2.5-ProSchema Generator=Llama-3.3-70B-Instruct, Approach=Evidence-Guided Schema Normalization2025.11 | 78.51 | 78.2 | |
| Llama-3.1-8B-InstructSchema Generator=Gemini 2.5 Flash, Approach=Evidence-Guided Schema Normalization2025.11 | 78.08 | 79.49 | |
| Qwen2.5-7B-InstructSchema Generator=Gemini 2.5 Flash, Approach=Evidence-Guided Schema Normalization2025.11 | 77.84 | 78.86 | |
| Qwen2.5-7B-InstructSchema Generator=Llama-3.3-70B-Instruct, Approach=Evidence-Guided Schema Normalization2025.11 | 75.3 | 75.07 | |
| Gemini-2.0-FlashSchema Generator=Llama-3.3-70B-Instruct, Approach=Evidence-Guided Schema Normalization2025.11 | 72.7 | 73.33 | |
| Llama-3.3-70B-InstructSchema Generator=Gemini 2.5 Flash, Approach=Evidence-Guided Schema Normalization2025.11 | 69.86 | 75.54 | |
| GPT-4o-miniSchema Generator=Gemini 2.5 Flash, Approach=Evidence-Guided Schema Normalization2025.11 | 69.56 | 66.8 | |
| Gemini-2.5-ProSchema Generator=IRE+CoT, Approach=Baseline2025.11 | 68.89 | 75.3 | |
| Gemini-2.5-ProSchema Generator=Gemini 2.5 Flash, Approach=Evidence-Guided Schema Normalization2025.11 | 68.25 | 67.18 | |
| Llama-3.3-70B-InstructSchema Generator=Llama-3.1-8B-Instruct, Approach=Evidence-Guided Schema Normalization2025.11 | 64.79 | 63.66 | |
| Gemini-2.5-ProSchema Generator=Llama-3.1-8B-Instruct, Approach=Evidence-Guided Schema Normalization2025.11 | 63.13 | 63.07 | |
| GPT-4o-miniSchema Generator=Llama-3.1-8B-Instruct, Approach=Evidence-Guided Schema Normalization2025.11 | 62.88 | 63.15 | |
| Llama-3.1-8B-InstructSchema Generator=Llama-3.1-8B-Instruct, Approach=Evidence-Guided Schema Normalization2025.11 | 61.78 | 60.66 | |
| Qwen2.5-7B-InstructSchema Generator=Llama-3.1-8B-Instruct, Approach=Evidence-Guided Schema Normalization2025.11 | 60.91 | 60.32 | |
| Gemini-2.0-FlashSchema Generator=Llama-3.1-8B-Instruct, Approach=Evidence-Guided Schema Normalization2025.11 | 60.37 | 60.32 | |
| Gemini-2.0-FlashSchema Generator=IRE+CoT, Approach=Baseline2025.11 | 48.98 | 55.93 | |
| GPT-4o-miniSchema Generator=IRE+CoT, Approach=Baseline2025.11 | 44.53 | 52.27 | |
| Llama-3.3-70B-InstructSchema Generator=IRE+CoT, Approach=Baseline2025.11 | 41.08 | 54.91 | |
| Qwen2.5-7B-InstructSchema Generator=IRE+CoT, Approach=Baseline2025.11 | 30.4 | 29.22 | |
| Llama-3.1-8B-InstructSchema Generator=IRE+CoT, Approach=Baseline2025.11 | 29.83 | 37.95 |