Image Captioning on PixMoCap (test)
70.8CapArena Win Rate (v.s. Base Model)RubiCap-7B-PixMoCap
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| RubiCap-7B-PixMoCapFine-tuning=GRPO, Data=50k PixMoCap2026.03 | 70.8 | 66 | 59.42 | 75.47 | 23.73 | 19.93 | 7.53 | 46.13 | |
| RubiCap-3B-PixMoCapFine-tuning strategy=GRPO, Training sample size=50k, Training Dataset=PixMoCap Dataset, Reward signal type=RubiCap2026.03 | 68.6 | 42.8 | 57.89 | 69.49 | 23.3 | 18.16 | 8.05 | 41.18 | |
| Reference-LikertFine-tuning=GRPO, Data=50k PixMoCap2026.03 | 67.2 | 59.4 | 59.33 | 74.54 | 22.47 | 20.26 | 6.56 | 44.25 | |
| Direct-LikertFine-tuning=GRPO, Data=50k PixMoCap2026.03 | 66.2 | 61 | 59.3 | 75.22 | 23.6 | 19.93 | 7.55 | 44.69 | |
| Direct-LikertFine-tuning strategy=GRPO, Training sample size=50k, Training Dataset=PixMoCap Dataset, Reward signal type=Direct-Likert2026.03 | 64 | 37.4 | 58.35 | 69.73 | 23.11 | 18.36 | 7.59 | 39.79 | |
| RubiCap-2B-PixMoCapFine-tuning Strategy=GRPO, Training Dataset=PixMoCap2026.03 | 61.6 | 34.6 | 57.19 | 64.4 | 23.08 | 16.17 | 7.75 | 37.83 | |
| NLP Metric-based RLFine-tuning=GRPO, Data=50k PixMoCap2026.03 | 59.6 | 56.2 | 59.66 | 72.38 | 24.71 | 18.71 | 9.47 | 42.96 | |
| CapRL-3B2026.03 | 59.4 | 34.4 | 59.07 | 0 | 18.46 | 20.75 | 4.08 | 28.02 | |
| Direct-LikertFine-tuning Strategy=GRPO, Training Dataset=PixMoCap2026.03 | 57.2 | 31.4 | 57.07 | 64.57 | 22.87 | 16.35 | 7.51 | 36.71 | |
| NLP Metric-based RLFine-tuning strategy=GRPO, Training sample size=50k, Training Dataset=PixMoCap Dataset, Reward signal type=NLP Metric2026.03 | 54 | 35.8 | 58.12 | 71.22 | 24.31 | 17.83 | 9.14 | 38.63 | |
| NLP Metric-based RLFine-tuning Strategy=GRPO, Training Dataset=PixMoCap2026.03 | 52.2 | 27 | 57.63 | 69.02 | 23.97 | 16.56 | 7.89 | 36.32 | |
| Base Model (Qwen2-VL-2B-Instruct)Backbone=Qwen2-VL-2B-Instruct2026.03 | 50 | 28.8 | 57.33 | 67.34 | 22.14 | 17.24 | 6.4 | 35.61 | |
| Base Model (Qwen2.5-VL-3B-Instruct)Backbone=Qwen2.5-VL-3B-Instruct2026.03 | 50 | 34 | 56.87 | 68.83 | 21.76 | 17.98 | 6.5 | 36.56 | |
| Base Model (Qwen2.5-VL-7B-Instruct)Protocol=Pre-trained, Backbone=Qwen2.5-VL-7B-Instruct2026.03 | 50 | 54 | 57.63 | 69.39 | 23.18 | 18.32 | 8.05 | 40.08 | |
| Reference-LikertFine-tuning Strategy=GRPO, Training Dataset=PixMoCap2026.03 | 9 | 2.6 | 25.72 | 22.43 | 12.03 | 4.72 | 1.06 | 11.08 | |
| Reference-LikertFine-tuning strategy=GRPO, Training sample size=50k, Training Dataset=PixMoCap Dataset, Reward signal type=Reference-Likert2026.03 | 7.8 | 4 | 13.33 | 16.8 | 7.27 | 2.13 | 0.01 | 7.33 |