Optical Illusion Character Recognition on IlluChar (Noise)
98.9L ScoreHuman
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Human2026.03 | 98.9 | 100 | 100 | 99.5 | — | |
| SMSP (Ours)Model=Qwen3-VL-235B-A22B-Instruct2026.03 | 79.7 | 98 | 94 | 88.6 | — | |
| SMSP (Ours)Model=Qwen3-VL-8B-Instruct2026.03 | 79.1 | 95.4 | 93 | 87.3 | — | |
| SMSP (Ours)Model=GLM-4.5V2026.03 | 69.9 | 97.4 | 92.1 | 83.5 | — | |
| SMSP (Ours)Model=Gemini-2.5-Pro2026.03 | 55.9 | 91.6 | 85.6 | 73.8 | — | |
| Blur with HistogramModel=GLM-4.5V2026.03 | 45.4 | 84.7 | 6 | 45.4 | — | |
| Blur with HistogramModel=Gemini-2.5-Pro2026.03 | 45.2 | 18.9 | 68.9 | 44.5 | — | |
| SMSP (Ours)Model=GPT-5.22026.03 | 44.8 | 85.1 | 70.6 | 62.9 | — | |
| Blur with HistogramModel=Qwen3-VL-235B-A22B-Instruct2026.03 | 42.9 | 80 | 3.1 | 42.2 | — | |
| Blur with HistogramModel=Qwen3-VL-8B-Instruct2026.03 | 42.6 | 66 | 1.4 | 37.7 | — | |
| SMSP (Ours)Model=Claude-Sonnet-4.52026.03 | 41.3 | 80.6 | 65.4 | 58.6 | — | |
| Blur with HistogramModel=GPT-5.22026.03 | 33.4 | 58.7 | 2.9 | 32 | — | |
| FilteredModel=GLM-4.5V2026.03 | 27.3 | 97.1 | 51.7 | 53.1 | — | |
| FilteredModel=Gemini-2.5-Pro2026.03 | 26.5 | 85.6 | 43.7 | 47.3 | — | |
| Blur with HistogramModel=Claude-Sonnet-4.52026.03 | 25.8 | 60 | 3.6 | 29.1 | — | |
| FilteredModel=Qwen3-VL-235B-A22B-Instruct2026.03 | 24.4 | 73.1 | 28.6 | 38.9 | — | |
| FilteredModel=Qwen3-VL-8B-Instruct2026.03 | 20.9 | 59.1 | 17.1 | 30.3 | — | |
| FilteredModel=GPT-5.22026.03 | 13.9 | 64.6 | 33.4 | 33.1 | — | |
| FilteredModel=Claude-Sonnet-4.52026.03 | 6.4 | 28 | 2.6 | 11.2 | — | |
| CoTModel=GPT-5.22026.03 | 3.3 | 2.9 | 26.7 | 9.6 | — | |
| VanillaModel=GPT-5.22026.03 | 3.1 | 2.9 | 26 | 9.3 | — | |
| GPT-5.22026.03 | 3.1 | 2.9 | 26 | 9.3 | -79.8 | |
| CoTModel=Qwen3-VL-235B-A22B-Instruct2026.03 | 1.7 | 15.9 | 52.9 | 19.6 | — | |
| VanillaModel=Gemini-2.5-Pro2026.03 | 1.7 | 17.1 | 74.4 | 25.8 | — | |
| Gemini-2.5-Pro2026.03 | 1.7 | 17.1 | 74.4 | 25.8 | -69.1 | |
| CoTModel=GLM-4.5V2026.03 | 1.5 | 24.3 | 74.7 | 27.7 | — | |
| CoTModel=Claude-Sonnet-4.52026.03 | 1.5 | 1.9 | 0 | 1.2 | — | |
| VanillaModel=Claude-Sonnet-4.52026.03 | 1.4 | 1.7 | 0.3 | 1.2 | — | |
| Claude-Sonnet-4.52026.03 | 1.4 | 1.7 | 0.3 | 1.2 | -86.3 | |
| VanillaModel=GLM-4.5V2026.03 | 1.3 | 26.3 | 75.7 | 28.5 | — | |
| GLM-4.5V2026.03 | 1.3 | 26.3 | 75.7 | 28.5 | -66.8 | |
| CoTModel=Gemini-2.5-Pro2026.03 | 1.2 | 17 | 68.9 | 24 | — | |
| CoTModel=Qwen3-VL-8B-Instruct2026.03 | 1 | 12.3 | 49.9 | 17.5 | — | |
| VanillaModel=Qwen3-VL-235B-A22B-Instruct2026.03 | 0.9 | 16.4 | 55.4 | 20 | — | |
| Qwen3-VL-235B-A22B2026.03 | 0.9 | 16.4 | 55.4 | 20 | -77.7 | |
| VanillaModel=Qwen3-VL-8B-Instruct2026.03 | 0.6 | 11.4 | 48.7 | 16.7 | — | |
| Qwen3-VL-8B2026.03 | 0.6 | 11.4 | 48.7 | 16.7 | -76.5 |