Temporal Reasoning on GDELT (test)
63.77Positive AccuracyGETER
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GETERBackbone=Qwen2.5-7B-Instruct2025.05 | 63.77 | 70.06 | 88.42 | 73.27 | |
| Llama3-8B-InstructProtocol=LoRA, Reasoning Chains=true2025.05 | 62.86 | 28.57 | 78.56 | 56.44 | |
| GETERBackbone=Llama3-8B-Instruct2025.05 | 62.62 | 68.74 | 88.73 | 72.51 | |
| Llama3-8B-InstructProtocol=LoRA, Reasoning Chains=false2025.05 | 61.94 | 7.19 | 69.14 | 46.29 | |
| GETERBackbone=Mistral-7B-Instruct2025.05 | 61.29 | 68.92 | 88.59 | 72.02 | |
| Mistral-7B-InstructProtocol=LoRA, Reasoning Chains=true2025.05 | 60.56 | 55.09 | 81.29 | 65.05 | |
| Llama3-8B-InstructProtocol=zero-shot, Reasoning Chains=true2025.05 | 53.48 | 15.62 | 29.47 | 33.9 | |
| GPT-4oProtocol=zero-shot, Reasoning Chains=true2025.05 | 42.74 | 37.16 | 29.21 | 36.83 | |
| Qwen2.5-7B-InstructProtocol=LoRA, Reasoning Chains=false2025.05 | 31.28 | 52.11 | 12.41 | 32.36 | |
| Qwen2.5-7B-InstructProtocol=LoRA, Reasoning Chains=true2025.05 | 22.39 | 56.61 | 66.79 | 46.95 | |
| Qwen2.5-7B-InstructProtocol=zero-shot, Reasoning Chains=true2025.05 | 22.22 | 48.23 | 1.21 | 24.34 | |
| Mistral-7B-InstructProtocol=LoRA, Reasoning Chains=false2025.05 | 19.45 | 58.16 | 71.52 | 47.8 | |
| GPT-4oProtocol=zero-shot, Reasoning Chains=false2025.05 | 19.08 | 43.78 | 25.5 | 29.06 | |
| Qwen2.5-7B-InstructProtocol=zero-shot, Reasoning Chains=false2025.05 | 11.27 | 44.92 | 19.81 | 24.81 | |
| Mistral-7B-InstructProtocol=zero-shot, Reasoning Chains=false2025.05 | 5.52 | 40.64 | 23.5 | 22.39 | |
| Llama3-8B-InstructProtocol=zero-shot, Reasoning Chains=false2025.05 | 1.95 | 33.13 | 39.44 | 23.44 | |
| Mistral-7B-InstructProtocol=zero-shot, Reasoning Chains=true2025.05 | 0.99 | 24.93 | 43.23 | 21.55 |