Temporal Reasoning on ICEWS05-15 (test)
78.94Positive ScoreGETER
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GETERBackbone=Mistral-7B-Instruct2025.05 | 78.94 | 76.48 | 90.38 | 81.8 | |
| GETERBackbone=Llama3-8B-Instruct2025.05 | 78.58 | 75.95 | 91.48 | 81.84 | |
| GETERBackbone=Qwen2.5-7B-Instruct2025.05 | 78.23 | 72.95 | 89.9 | 80.23 | |
| Mistral-7B-InstructProtocol=LoRA, Reasoning Chains=true2025.05 | 72.53 | 71.95 | 84.18 | 76.07 | |
| Llama3-8B-InstructProtocol=LoRA, Reasoning Chains=true2025.05 | 71.32 | 51.77 | 74.4 | 65.86 | |
| Mistral-7B-InstructProtocol=LoRA, Reasoning Chains=false2025.05 | 70.81 | 39.12 | 75.8 | 61.95 | |
| Qwen2.5-7B-InstructProtocol=LoRA, Reasoning Chains=true2025.05 | 66.83 | 70.95 | 84.09 | 73.72 | |
| Llama3-8B-InstructProtocol=LoRA, Reasoning Chains=false2025.05 | 65.67 | 38.56 | 68.02 | 57.47 | |
| GPT-4oProtocol=zero-shot, Reasoning Chains=true2025.05 | 61.63 | 11.89 | 47.16 | 40.58 | |
| Llama3-8B-InstructProtocol=zero-shot, Reasoning Chains=true2025.05 | 57.14 | 17.5 | 14.03 | 30.24 | |
| GPT-4oProtocol=zero-shot, Reasoning Chains=false2025.05 | 55.45 | 26.33 | 15.47 | 33.03 | |
| Qwen2.5-7B-InstructProtocol=LoRA, Reasoning Chains=false2025.05 | 55.33 | 68.65 | 85.89 | 69.52 | |
| Qwen2.5-7B-InstructProtocol=zero-shot, Reasoning Chains=true2025.05 | 40.81 | 48.32 | 1.75 | 30.78 | |
| Qwen2.5-7B-InstructProtocol=zero-shot, Reasoning Chains=false2025.05 | 31.71 | 39.45 | 15.82 | 29.17 | |
| Mistral-7B-InstructProtocol=zero-shot, Reasoning Chains=true2025.05 | 17.73 | 29.8 | 49.69 | 31.96 | |
| Llama3-8B-InstructProtocol=zero-shot, Reasoning Chains=false2025.05 | 11.75 | 28.98 | 39.41 | 26.3 | |
| Mistral-7B-InstructProtocol=zero-shot, Reasoning Chains=false2025.05 | 7.56 | 33.21 | 46.1 | 28.37 |