Open-ended event forecasting on OpenForecast
18.16SP@1GRPO
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GRPOBackbone=Qwen3-7B-Instruct2026.04 | 18.16 | — | — | 24.56 | 9.41 | 13.37 | 0.64 | |
| SCATTERBackbone=Qwen3-7B-Instruct2026.04 | 17.65 | — | — | 43.64 | 9.46 | 25.48 | 10.95 | |
| Qwen2.5-3B-Instruct + SCATTERBackbone=Qwen2.5-3B-Instruct, Alignment Strategy=+ SCATTER2026.04 | 17.61 | 8.55 | 30.26 | 41.89 | 9.77 | 24.29 | 9.14 | |
| Qwen2.5-3B-InstructAlignment Strategy=+ GRPO2026.04 | 17.6 | 11.84 | 25.66 | 27.85 | 9.7 | 15.94 | 0.87 | |
| Llama3.2-3B-Instruct + SCATTERBackbone=Llama3.2-3B-Instruct, Alignment Strategy=+ SCATTER2026.04 | 17.45 | 7.73 | 31.36 | 42.32 | 9.47 | 25.64 | 6.54 | |
| Llama3.2-3B-InstructAlignment Strategy=+ GRPO2026.04 | 13.72 | 11.9 | 28.29 | 21.05 | 7.08 | 11.61 | 0.71 | |
| SFTBackbone=Qwen3-7B-Instruct2026.04 | 10.35 | — | — | 28.95 | 5.34 | 16.31 | 8.57 | |
| Llama3.2-3B-InstructAlignment Strategy=Base2026.04 | 9.4 | 4.4 | 21.71 | 27.19 | 4.85 | 15.42 | 5.73 | |
| GPT-4o-mini2026.04 | 9.39 | 7.55 | 25.44 | 22.81 | 4.96 | 12.76 | 4.23 | |
| BaseBackbone=Qwen3-7B-Instruct2026.04 | 9.33 | — | — | 23.9 | 4.69 | 13.04 | 4.44 | |
| Qwen2.5-3B-InstructAlignment Strategy=+ SFT2026.04 | 8.55 | 5.51 | 23.68 | 24.56 | 4.48 | 13.85 | 7.1 | |
| Llama3.2-3B-InstructAlignment Strategy=+ SFT2026.04 | 7.1 | 4.5 | 20.39 | 21.27 | 3.64 | 11.72 | 7.74 | |
| Qwen2.5-3B-InstructAlignment Strategy=Base2026.04 | 6.95 | 5.96 | 25.88 | 20.61 | 3.67 | 10.72 | 5.07 |