Preference Alignment on Koala
196Wins (Count)CLIPer
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CLIPerLabeler=Total2026.05 | 196 | 65.3 | |
| Preference PromptingLabeler=Total2026.05 | 114 | — | |
| CLIPerLabeler=Labeler 52026.05 | 45 | 90 | |
| CLIPerLabeler=Labeler 22026.05 | 43 | 86 | |
| CLIPerLabeler=Labeler 12026.05 | 31 | 62 | |
| CLIPerLabeler=Labeler 62026.05 | 28 | 56 | |
| Preference PromptingLabeler=Labeler 32026.05 | 26 | — | |
| CLIPerLabeler=Labeler 42026.05 | 25 | 50 | |
| Preference PromptingLabeler=Labeler 42026.05 | 25 | — | |
| CLIPerLabeler=Labeler 32026.05 | 24 | 48 | |
| Preference PromptingLabeler=Labeler 62026.05 | 22 | — | |
| Preference PromptingLabeler=Labeler 12026.05 | 19 | — | |
| Preference PromptingLabeler=Labeler 22026.05 | 17 | — | |
| Preference PromptingLabeler=Labeler 52026.05 | 5 | — | |
| CLIPer (Vanilla Baseline)Evaluator=Reward Model, Preference Dimensions=3-dim2026.05 | — | 90 | |
| CLIPer (Vanilla Baseline)Evaluator=GPT-4o-mini, Preference Dimensions=3-dim2026.05 | — | 80.67 | |
| Direct PromptingEvaluator=Reward Model, Preference Dimensions=3-dim2026.05 | — | 59.33 | |
| Direct PromptingEvaluator=GPT-4o-mini, Preference Dimensions=3-dim2026.05 | — | 53.67 | |
| p-soup & Direct Fine-tuningEvaluator=Reward Model, Preference Dimensions=3-dim2026.05 | — | 85.33 | |
| p-soup & Direct Fine-tuningEvaluator=GPT-4o-mini, Preference Dimensions=3-dim2026.05 | — | 54.67 |