Sarcasm Detection on MMSD 1.0 (test)
86.7F1 ScoreCommander
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CommanderModel=BERT+ViT, Parameters=110M2025.06 | 86.7 | 90.1 | 87.4 | 86.1 | |
| CommanderModel=GPT-4o, Parameters=300B2025.06 | 81.4 | 83.1 | 78.9 | 84.2 | |
| BERT (Fine-tuned)Model=BERT+ViT, Parameters=110M2025.06 | 80.8 | 82.6 | 81.2 | 80.1 | |
| S^3 AgentModel=GPT-4o, Parameters=300B2025.06 | 77.9 | 80.2 | 74.6 | 81.5 | |
| Generated Knowledge PromptingModel=GPT-4o, Parameters=300B2025.06 | 76.3 | 79.7 | 73.5 | 79.2 | |
| Automatic Prompt EngineerModel=GPT-4o, Parameters=300B2025.06 | 75.8 | 79.1 | 72.9 | 78.9 | |
| Zero-shot CoTModel=GPT-4o, Parameters=300B2025.06 | 74.2 | 78.5 | 71.3 | 77.4 | |
| CommanderModel=Gemini Pro, Parameters=600B2025.06 | 73.8 | 76.2 | 70.1 | 78.3 | |
| Plan-and-SolveModel=GPT-4o, Parameters=300B2025.06 | 73.6 | 77.8 | 70.2 | 77.1 | |
| CommanderModel=MiniCPM-V2, Parameters=2.8B2025.06 | 72.5 | 74.5 | 65.9 | 80.8 | |
| S^3 AgentModel=Gemini Pro, Parameters=600B2025.06 | 70.1 | 74.9 | 67.8 | 75.7 | |
| Generated Knowledge PromptingModel=Gemini Pro, Parameters=600B2025.06 | 69.5 | 73.6 | 66.3 | 74.2 | |
| CommanderModel=Qwen-VL-Chat, Parameters=9B2025.06 | 69.3 | 68.2 | 58 | 86.2 | |
| Automatic Prompt EngineerModel=Gemini Pro, Parameters=600B2025.06 | 68.7 | 72.8 | 64.1 | 73.9 | |
| S^3 AgentModel=Qwen-VL-Chat, Parameters=9B2025.06 | 68.1 | 67.5 | 57.6 | 83.3 | |
| Zero-shot CoTModel=Gemini Pro, Parameters=600B2025.06 | 67.2 | 71.4 | 62.8 | 70.5 | |
| Zero-shot CoTModel=Qwen-VL-Chat, Parameters=9B2025.06 | 66.4 | 66 | 56.5 | 80.7 | |
| Plan-and-SolveModel=Gemini Pro, Parameters=600B2025.06 | 66.3 | 70.2 | 62.5 | 71.8 | |
| Automatic Prompt EngineerModel=Qwen-VL-Chat, Parameters=9B2025.06 | 64.5 | 60.5 | 51.6 | 86 | |
| Automatic Prompt EngineerModel=MiniCPM-V2, Parameters=2.8B2025.06 | 63.2 | 69.2 | 63.1 | 63.4 | |
| Generated Knowledge PromptingModel=MiniCPM-V2, Parameters=2.8B2025.06 | 63.2 | 67.5 | 59.9 | 66.8 | |
| Zero-shot CoTModel=MiniCPM-V2, Parameters=2.8B2025.06 | 61.4 | 68.1 | 62.1 | 60.7 | |
| CommanderModel=DeepSeek-VL-Chat, Parameters=7B2025.06 | 61.1 | 69.4 | 64.9 | 57.7 | |
| Plan-and-SolveModel=MiniCPM-V2, Parameters=2.8B2025.06 | 60.9 | 66 | 58.5 | 63.5 | |
| S^3 AgentModel=MiniCPM-V2, Parameters=2.8B2025.06 | 60.8 | 66.6 | 59.6 | 62 | |
| S^3 AgentModel=DeepSeek-VL-Chat, Parameters=7B2025.06 | 59.7 | 45.3 | 43.1 | 59.7 | |
| Plan-and-SolveModel=Qwen-VL-Chat, Parameters=9B2025.06 | 59.4 | 64.8 | 57.2 | 61.8 | |
| CommanderModel=Yi-VL, Parameters=6B2025.06 | 59.1 | 70.8 | 71.2 | 50.5 | |
| Generated Knowledge PromptingModel=DeepSeek-VL-Chat, Parameters=7B2025.06 | 55.6 | 56.8 | 48.7 | 65 | |
| Automatic Prompt EngineerModel=DeepSeek-VL-Chat, Parameters=7B2025.06 | 55.1 | 68 | 66.4 | 47.6 | |
| Plan-and-SolveModel=DeepSeek-VL-Chat, Parameters=7B2025.06 | 54.9 | 61.9 | 54.3 | 55.5 | |
| Zero-shot CoTModel=DeepSeek-VL-Chat, Parameters=7B2025.06 | 54.4 | 65.3 | 60.1 | 49.8 | |
| Generated Knowledge PromptingModel=Yi-VL, Parameters=6B2025.06 | 52.7 | 55.7 | 47.5 | 59.2 | |
| S^3 AgentModel=Yi-VL, Parameters=6B2025.06 | 52.3 | 41.4 | 39.6 | 77 | |
| Plan-and-SolveModel=Yi-VL, Parameters=6B2025.06 | 51.3 | 62.6 | 56.2 | 47.2 | |
| Generated Knowledge PromptingModel=Qwen-VL-Chat, Parameters=9B2025.06 | 50.9 | 67.1 | 64.7 | 40.9 | |
| Automatic Prompt EngineerModel=Yi-VL, Parameters=6B2025.06 | 49.4 | 53.3 | 45 | 54.7 | |
| Zero-shot CoTModel=Yi-VL, Parameters=6B2025.06 | 45.8 | 59.2 | 51.4 | 41.4 |