Text-to-Text Creative Reasoning on Oogiri English T2T (test)
52.93T1 AccuracyCogVLM-17B+CLoT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CogVLM-17B+CLoTSize=7B2023.12 | 52.9 | 33.6 | 21.8 | 68.6 | 44.2 | |
| Qwen-VL+CLoTSize=7B2023.12 | 51.7 | 32.3 | 24.8 | 65 | 43.4 | |
| GPT-42023.12 | 49.2 | 20.4 | 3.6 | 54.7 | 32 | |
| GPT-3.52023.12 | 45.3 | 30.4 | 6.7 | 61.6 | 36 | |
| Vicuna-v1.5Size=7B2023.12 | 32.6 | 23.5 | 0 | 63 | 29.8 | |
| Vicuna-v1.5Size=13B2023.12 | 30.2 | 23 | 2.7 | 62.2 | 29.5 | |
| Baichuan2Size=7B2023.12 | 28.3 | 22.6 | 11.6 | 64.6 | 31.8 | |
| LLAMA2Size=70B2023.12 | 27.8 | 16.1 | 3.8 | 62 | 27.4 | |
| QwenSize=14B2023.12 | 27.4 | 22.2 | 12.3 | 59.5 | 30.3 | |
| QwenSize=7B2023.12 | 23.1 | 20.4 | 8 | 61.4 | 28.2 | |
| Baichuan2Size=13B2023.12 | 21.7 | 18.3 | 8.9 | 61.5 | 27.6 | |
| LLAMA2Size=7B2023.12 | 18.9 | 13.5 | 1.1 | 60.4 | 23.5 | |
| LLAMA2Size=13B2023.12 | 15.6 | 20 | 1.8 | 60.5 | 24.5 | |
| ChatGLM3Size=6B2023.12 | 15.6 | 17 | 5.4 | 59.4 | 24.3 |