Text-to-Image Generation on Pick-a-Pic (test)
87PickScoreSPIN-Diffusion-Iter3
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SPIN-Diffusion-Iter3iteration=3, baseline_comparison=winning rate over SD-1.52024.02 | 87 | 86.2 | 77 | 93.8 | 91.6 | — | |
| SPIN-Diffusion-Iter2iteration=2, baseline_comparison=winning rate over SD-1.52024.02 | 86.6 | 82.6 | 72.6 | 92.2 | 90 | — | |
| Diffusion-DPO (ours)baseline_comparison=winning rate over SD-1.52024.02 | 83.8 | 81.2 | 71.2 | 69 | 84.8 | — | |
| Diffusion-DPObaseline_comparison=winning rate over SD-1.52024.02 | 78.4 | 75.8 | 65 | 65.4 | 79.8 | — | |
| SPIN-Diffusion-Iter1iteration=1, baseline_comparison=winning rate over SD-1.52024.02 | 75.4 | 70 | 65.8 | 86 | 80.8 | — | |
| SPIN-Diffusioniteration=3, evaluation_protocol=winning rate over Diffusion DPO (ours)2024.02 | 62.4 | 56.2 | 55.8 | 86.8 | 70.2 | — | |
| SFTbaseline_comparison=winning rate over SD-1.52024.02 | 62.4 | 82 | 75 | 70.8 | 73.2 | — | |
| SPIN-Diffusioniteration=2, evaluation_protocol=winning rate over Diffusion DPO (ours)2024.02 | 56.8 | 49 | 52.6 | 86.6 | 68.2 | — | |
| SPIN-Diffusioniteration=1, evaluation_protocol=winning rate over Diffusion DPO (ours)2024.02 | 37.2 | 35.6 | 40.6 | 74.8 | 47.4 | — | |
| Diffusion-DPOevaluation_protocol=winning rate over Diffusion DPO (ours)2024.02 | 30.6 | 29.4 | 36.8 | 45.2 | 30.4 | — | |
| SFTevaluation_protocol=winning rate over Diffusion DPO (ours)2024.02 | 26.8 | 48.2 | 51.4 | 52.8 | 35.2 | — | |
| SPIN-Diffusioniteration=32024.02 | 22.0703 | 0.2761 | 1.3073 | 6.2769 | 7.4827 | — | |
| SIPOModel=SIPO (ours)2025.05 | 22.013 | 0.2763 | 1.1257 | 6.2451 | 7.415 | — | |
| SPIN-Diffusioniteration=22024.02 | 22.0101 | 0.2751 | 1.3059 | 6.2385 | 7.4574 | — | |
| SPIN-DiffusionIteration=32024.02 | 22.0024 | 0.2759 | 1.1239 | 6.2481 | 7.4126 | — | |
| SPIN-DiffusionModel=SPIN-Diffusion2025.05 | 22.0024 | 0.2759 | 1.1239 | 6.2481 | 7.4126 | — | |
| SPIN-DiffusionIteration=22024.02 | 21.9567 | 0.2751 | 1.1086 | 6.2399 | 7.3951 | — | |
| Diffusion-DPOimplementation=this paper2024.02 | 21.8995 | 0.2756 | 1.2219 | 5.8895 | 7.3216 | — | |
| Diffusion-DPOimplementation=ours2024.02 | 21.8866 | 0.2753 | 1.0495 | 5.8918 | 7.2758 | — | |
| Diffusion-DPOModel=Diffusion-DPO2025.05 | 21.8866 | 0.2753 | 1.0495 | 5.8918 | 7.2758 | — | |
| DPO-C&MModel=DPO-C&M2025.05 | 21.8841 | 0.2749 | 1.1753 | 6.1397 | 7.3683 | — | |
| SPIN-DiffusionIteration=12024.02 | 21.6651 | 0.2728 | 1.0131 | 6.1206 | 7.2679 | — | |
| SPIN-Diffusioniteration=12024.02 | 21.6464 | 0.2739 | 1.1366 | 6.1297 | 7.2967 | — | |
| Diffusion-DPOimplementation=original2024.02 | 21.6064 | 0.2729 | 1.1361 | 5.8837 | 7.2248 | — | |
| Diffusion-DPO2024.02 | 21.5919 | 0.2724 | 0.9625 | 5.8635 | 7.1726 | — | |
| SFTimplementation=ours2024.02 | 21.4542 | 0.2749 | 1.1051 | 5.9451 | 7.1948 | — | |
| SFTModel=SFT2025.05 | 21.4542 | 0.2749 | 1.1051 | 5.9451 | 7.1948 | — | |
| SFTimplementation=this paper2024.02 | 21.4159 | 0.275 | 1.3161 | 5.9331 | 7.235 | — | |
| SD-1.52024.02 | 21.1983 | 0.2699 | 0.8159 | 5.7691 | 7.0133 | — | |
| SD-1.5Model=SD-1.52025.05 | 21.1983 | 0.2699 | 0.8159 | 5.7691 | 7.0133 | — | |
| SD-1.52024.02 | 21.1813 | 0.2705 | 0.9184 | 5.7726 | 7.0357 | — | |
| TGOBackbone=SD v1.52026.05 | 21.05 | 0.286 | 0.6703 | 5.55 | — | 27.62 | |
| Diffusion-KTOBackbone=SD v1.52026.05 | 20.94 | 0.2814 | 0.6381 | 5.51 | — | 27.48 | |
| Diffusion-DPOBackbone=SD v1.52026.05 | 20.78 | 0.2594 | 0.3433 | 5.38 | — | 27.45 | |
| SFTBackbone=SD v1.52026.05 | 20.68 | 0.2724 | 0.5015 | 5.53 | — | 27.04 | |
| CSFTBackbone=SD v1.52026.05 | 20.48 | 0.2649 | 0.3473 | 5.49 | — | 26.83 | |
| SD v1.5Backbone=SD v1.52026.05 | 20.35 | 0.2469 | 0.1131 | 5.33 | — | 26.84 | |
| AlignPropBackbone=SD v1.52026.05 | 20.35 | 0.2469 | 0.1131 | 5.33 | — | 26.84 | |
| DSPOBackbone=SD v1.52026.05 | 20.35 | 0.2469 | 0.1131 | 5.33 | — | 26.84 | |
| Golden NoiseBackbone=SDXL2026.04 | 17.9 | 28.08 | 0.76 | 6.05 | — | 73.87 | |
| Oracle NoiseBackbone=SDXL2026.04 | 17.85 | 27.97 | 0.67 | 6.13 | — | 73 | |
| GaussianBackbone=SDXL2026.04 | 17.73 | 27.88 | 0.58 | 6.03 | — | 71.83 | |
| SD-1.5evaluation_protocol=winning rate over Diffusion DPO (ours)2024.02 | 16.2 | 20.8 | 28.8 | 31 | 15.2 | — |