Weather Forecasting Report Generation on WSInstruct (test)
44BLEU-1WeatherSyn-DPO
Evaluation results
| Method | Links | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| WeatherSyn-DPOAccess=Open-source, Backbone=Qwen3-VL-8B, Optimization=DPO2026.05 | 44 | 32 | 25 | 62 | 59 | 63 | 59 | 33 | 32 | 29 | 30 | — | — | — | — | — | — | — | — | — | |
| WeatherSynAccess=Open-source, Backbone=Qwen3-VL-8B2026.05 | 43 | 31 | 25 | 60 | 55 | 60 | 54 | 11 | 11 | 11 | 12 | — | — | — | — | — | — | — | — | — | |
| WeatherSyn-RFTAccess=Open-source, Backbone=Qwen3-VL-8B, Optimization=RFT2026.05 | 43 | 31 | 25 | 62 | 59 | 61 | 59 | 16 | 15 | 17 | 15 | — | — | — | — | — | — | — | — | — | |
| Gemini-3 Pro PreviewAccess=Closed-source, Few-shot=2-shot2026.05 | 37 | 28 | 23 | 63 | 60 | 63 | 60 | 24 | 27 | 29 | 31 | — | — | — | — | — | — | — | — | — | |
| GPT-4.1-MiniAccess=Closed-source, Few-shot=2-shot2026.05 | 20 | 15 | 13 | 49 | 49 | 49 | 48 | 1 | 1 | 1 | 1 | — | — | — | — | — | — | — | — | — | |
| WeatherQAAccess=Open-source, Backbone=Qwen3-VL-8B2026.05 | 19 | 15 | 14 | 50 | 36 | 51 | 36 | 1 | 1 | 1 | 1 | — | — | — | — | — | — | — | — | — | |
| OmniEarthAccess=Open-source, Backbone=Qwen3-VL-8B2026.05 | 17 | 17 | 14 | 51 | 40 | 51 | 40 | 1 | 1 | 1 | 1 | — | — | — | — | — | — | — | — | — | |
| GPT-5-NanoAccess=Closed-source, Few-shot=2-shot2026.05 | 16 | 13 | 10 | 50 | 47 | 50 | 48 | 3 | 3 | 2 | 2 | — | — | — | — | — | — | — | — | — | |
| GPT-5.2 ThinkingAccess=Closed-source, Few-shot=2-shot2026.05 | 12 | 12 | 11 | 51 | 49 | 52 | 49 | 6 | 7 | 7 | 5 | — | — | — | — | — | — | — | — | — | |
| Claude-3.7-SonnetAccess=Closed-source, Few-shot=2-shot2026.05 | 9 | 10 | 17 | 53 | 51 | 52 | 50 | 2 | 1 | 1 | 1 | — | — | — | — | — | — | — | — | — | |
| Gemini-2.5-FlashAccess=Closed-source, Few-shot=zero-shot2026.05 | 7 | 11 | 11 | 51 | 44 | 51 | 44 | 2 | 1 | 1 | 1 | — | — | — | — | — | — | — | — | — | |
| Claude-3.7-Sonnetshot=2-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | 39 | 55 | 45 | 31 | 58 | 66 | 67 | 69 | 53 | |
| Gemini-2.5-flashshot=zero-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | 35 | 55 | 44 | 33 | 59 | 52 | 60 | 76 | 51 | |
| GPT-4.1-minishot=2-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | 35 | 52 | 37 | 29 | 62 | 56 | 53 | 71 | 49 | |
| GPT-5-nanoshot=2-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | 36 | 53 | 37 | 32 | 63 | 48 | 53 | 78 | 50 | |
| OmniEarth2026.05 | — | — | — | — | — | — | — | — | — | — | — | 35 | 57 | 42 | 35 | 72 | 56 | 52 | 63 | 51 | |
| WeatherQA2026.05 | — | — | — | — | — | — | — | — | — | — | — | 34 | 55 | 49 | 47 | 60 | 47 | 49 | 64 | 50 | |
| WeatherSyn2026.05 | — | — | — | — | — | — | — | — | — | — | — | 42 | 62 | 63 | 43 | 75 | 64 | 67 | 66 | 60 | |
| WeatherSyn-DPO2026.05 | — | — | — | — | — | — | — | — | — | — | — | 44 | 63 | 67 | 49 | 77 | 64 | 72 | 67 | 62 | |
| WeatherSyn-RFT2026.05 | — | — | — | — | — | — | — | — | — | — | — | 44 | 62 | 69 | 47 | 75 | 64 | 74 | 66 | 62 |