Content Moderation on Hateful Memes seen (test)
89.2AUC-ROCPALI-X-VPD (specialist w/ CoT)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| PALI-X-VPD (specialist w/ CoT)Setting=Supervised, Protocol=Specialist w/ CoT2023.12 | 89.2 | 80.8 | |
| PALI-X-VPD (label-only FT)Setting=Supervised, Protocol=Label-only fine-tuning2023.12 | 88 | 77.6 | |
| Previous SOTASetting=Supervised2023.12 | 86.7 | 78.8 | |
| FlamingoSetting=Supervised, Model Size=80B2023.12 | 86.6 | — | |
| HumanSetting=Human Performance2023.12 | 82.7 | 84.7 | |
| VisualBERTSetting=Supervised2023.12 | 75.4 | 69.5 | |
| Generated ProgramsSetting=Unsupervised / Zero-Shot2023.12 | 70.1 | 69.7 |