Temporal Reasoning on ICEWS14 (test)
77.45Positive ScoreGETER
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GETERBackbone=Mistral-7B-Instruct2025.05 | 77.45 | 75.73 | 85.15 | 79.08 | |
| GETERBackbone=Qwen2.5-7B-Instruct2025.05 | 76.41 | 74.61 | 84.49 | 78.12 | |
| GETERBackbone=Llama3-8B-Instruct2025.05 | 75.07 | 67.38 | 81.15 | 74.25 | |
| Qwen2.5-7B-InstructProtocol=LoRA, Reasoning Chains=true2025.05 | 74.6 | 65.64 | 75.62 | 71.9 | |
| Mistral-7B-InstructProtocol=LoRA, Reasoning Chains=true2025.05 | 72.96 | 66.49 | 74.28 | 71.18 | |
| Llama3-8B-InstructProtocol=LoRA, Reasoning Chains=true2025.05 | 70.37 | 58.06 | 67.99 | 65.59 | |
| Qwen2.5-7B-InstructProtocol=LoRA, Reasoning Chains=false2025.05 | 62.82 | 58.59 | 71.97 | 64.03 | |
| Llama3-8B-InstructProtocol=LoRA, Reasoning Chains=false2025.05 | 62.27 | 36.98 | 48.17 | 49.81 | |
| GPT-4oProtocol=zero-shot, Reasoning Chains=true2025.05 | 60.1 | 9.54 | 48.56 | 39.95 | |
| Mistral-7B-InstructProtocol=LoRA, Reasoning Chains=false2025.05 | 58.04 | 65.44 | 80.03 | 66.79 | |
| Llama3-8B-InstructProtocol=zero-shot, Reasoning Chains=true2025.05 | 56.51 | 10.2 | 6.2 | 26.7 | |
| GPT-4oProtocol=zero-shot, Reasoning Chains=false2025.05 | 53.13 | 20.02 | 12.95 | 30.61 | |
| Qwen2.5-7B-InstructProtocol=zero-shot, Reasoning Chains=true2025.05 | 53.08 | 45.32 | 11.41 | 38.59 | |
| Qwen2.5-7B-InstructProtocol=zero-shot, Reasoning Chains=false2025.05 | 23.61 | 42.54 | 14.73 | 27.39 | |
| Mistral-7B-InstructProtocol=zero-shot, Reasoning Chains=true2025.05 | 22.04 | 27.64 | 40.76 | 29.26 | |
| Llama3-8B-InstructProtocol=zero-shot, Reasoning Chains=false2025.05 | 21.69 | 27.11 | 35.42 | 27.42 | |
| Mistral-7B-InstructProtocol=zero-shot, Reasoning Chains=false2025.05 | 3.65 | 39.44 | 46.44 | 27.81 |