Visual Reasoning on Bongard-OpenWorld 500-sample
93.6Overall ScoreGemini 2.0
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Gemini 2.0Paradigm=Componential Analysis (CA)2025.01 | 93.6 | 90.8 | 96.4 | |
| GPT-4oParadigm=Componential Analysis (CA)2025.01 | 92.8 | 92.8 | 92.8 | |
| Human AverageEvaluation Context=across samples2025.01 | 91 | — | — | |
| GPT-4oParadigm=Deductive Rule Learning (DRL)2025.01 | 88 | 82.8 | 93.2 | |
| Pixtral-12BParadigm=Componential Analysis (CA)2025.01 | 87.2 | 88.8 | 85.6 | |
| Gemini 2.0Paradigm=Deductive Rule Learning (DRL)2025.01 | 86.8 | 85.7 | 87.9 | |
| Gemini 2.0Paradigm=Direct Visual Rule Learning (DVRL)2025.01 | 82.2 | 80.5 | 83.9 | |
| GPT-4oParadigm=Direct Visual Rule Learning (DVRL)2025.01 | 80 | 66.4 | 93.6 | |
| Llava-7BParadigm=Componential Analysis (CA)2025.01 | 66.2 | 65.2 | 67.2 | |
| Llama-Vision-90BParadigm=Componential Analysis (CA)2025.01 | 55.1 | 58.7 | 51.5 | |
| Llama-Vision-11BParadigm=Componential Analysis (CA)2025.01 | 53.4 | 42.8 | 64 | |
| Llava-Llama3-8BParadigm=Componential Analysis (CA)2025.01 | 53.2 | 93 | 13.1 |