Multiple-choice Question Answering on WeatherQA (test)
44.8Accuracy (500 hPa)Weather-R1-7B
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Weather-R1-7BEvaluation Protocol=Fine-tuned, Model Scale=7B, Optimization=LoCo-RFT2026.01 | 44.8 | 47.5 | 46.6 | 41.9 | 64.4 | 67.7 | 57.4 | 52.9 | |
| SFT-7BEvaluation Protocol=Fine-tuned, Model Scale=7B2026.01 | 43.8 | 42.9 | 49.9 | 40.6 | 62 | 58.4 | 50.8 | 49.8 | |
| RFT-7BEvaluation Protocol=Fine-tuned, Model Scale=7B2026.01 | 43.1 | 48.4 | 47.9 | 41.5 | 64.6 | 62.9 | 51.4 | 51.4 | |
| Qwen2.5-VL-3BEvaluation Protocol=Zero-shot, Model Scale=3B2026.01 | 38 | 45.5 | 36 | 23.5 | 52.5 | 60 | 44 | 42.8 | |
| LLaVA-v1.6-13BEvaluation Protocol=Zero-shot, Model Scale=13B2026.01 | 35.5 | 35 | 28 | 22 | 60 | 23.5 | 25 | 32.7 | |
| Qwen2.5-VL-7BEvaluation Protocol=Zero-shot, Model Scale=7B2026.01 | 35.5 | 39.5 | 30.5 | 35 | 57 | 56.5 | 47.5 | 43.1 | |
| LLaVA-v1.6-7BEvaluation Protocol=Zero-shot, Model Scale=7B2026.01 | 34.5 | 26.5 | 23 | 37 | 27.5 | 25 | 20 | 27.6 |