Optical Illusion Character Recognition on IlluChar (Origin)
99.6L ScoreSMSP (Ours)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SMSP (Ours)Model=Qwen3-VL-235B-A22B-Instruct2026.03 | 99.6 | 100 | 100 | 99.8 | |
| SMSP (Ours)Model=GLM-4.5V2026.03 | 97.8 | 100 | 98.6 | 98.6 | |
| FilteredModel=Qwen3-VL-235B-A22B-Instruct2026.03 | 97.4 | 100 | 96.4 | 97.9 | |
| FilteredModel=Gemini-2.5-Pro2026.03 | 97.4 | 100 | 87.9 | 95.5 | |
| SMSP (Ours)Model=Qwen3-VL-8B-Instruct2026.03 | 95.3 | 100 | 97.9 | 97.3 | |
| VanillaModel=Qwen3-VL-235B-A22B-Instruct2026.03 | 95.3 | 100 | 99.3 | 97.7 | |
| Qwen3-VL-235B-A22B2026.03 | 95.3 | 100 | 99.3 | 97.7 | |
| CoTModel=Qwen3-VL-235B-A22B-Instruct2026.03 | 94.8 | 100 | 100 | 97.7 | |
| Blur with HistogramModel=Qwen3-VL-235B-A22B-Instruct2026.03 | 94.8 | 99.3 | 21.4 | 76 | |
| FilteredModel=Qwen3-VL-8B-Instruct2026.03 | 94.4 | 99.3 | 97.9 | 96.7 | |
| FilteredModel=GLM-4.5V2026.03 | 94 | 94.3 | 81.4 | 90.6 | |
| VanillaModel=GLM-4.5V2026.03 | 93.1 | 97.1 | 97.1 | 95.3 | |
| GLM-4.5V2026.03 | 93.1 | 97.1 | 97.1 | 95.3 | |
| SMSP (Ours)Model=Gemini-2.5-Pro2026.03 | 92.7 | 99.3 | 97.9 | 95.9 | |
| CoTModel=GLM-4.5V2026.03 | 92.2 | 92.9 | 96.4 | 93.6 | |
| CoTModel=Gemini-2.5-Pro2026.03 | 92.2 | 98.6 | 97.9 | 95.5 | |
| VanillaModel=Gemini-2.5-Pro2026.03 | 91.4 | 98.6 | 97.1 | 94.9 | |
| Gemini-2.5-Pro2026.03 | 91.4 | 98.6 | 97.1 | 94.9 | |
| Blur with HistogramModel=Qwen3-VL-8B-Instruct2026.03 | 90.9 | 95 | 10 | 69.9 | |
| Blur with HistogramModel=GLM-4.5V2026.03 | 90.5 | 92.9 | 12.1 | 69.7 | |
| Blur with HistogramModel=Gemini-2.5-Pro2026.03 | 88.8 | 98.6 | 97.9 | 93.9 | |
| SMSP (Ours)Model=GPT-5.22026.03 | 87.9 | 99.3 | 98.6 | 93.9 | |
| VanillaModel=Qwen3-VL-8B-Instruct2026.03 | 87.1 | 99.3 | 97.1 | 93.2 | |
| Qwen3-VL-8B2026.03 | 87.1 | 99.3 | 97.1 | 93.2 | |
| CoTModel=Qwen3-VL-8B-Instruct2026.03 | 86.6 | 99.3 | 97.9 | 93.2 | |
| SMSP (Ours)Model=Claude-Sonnet-4.52026.03 | 86.6 | 99.3 | 98.6 | 93.4 | |
| FilteredModel=Claude-Sonnet-4.52026.03 | 79.3 | 97.9 | 65.7 | 80.7 | |
| SMSP (Ours)Model=Qwen3-VL-235B-A22B-Instruct2026.03 | 78.2 | 89.7 | 94.7 | 82.4 | |
| VanillaModel=GPT-5.22026.03 | 77.2 | 99.3 | 98.6 | 89.1 | |
| CoTModel=GPT-5.22026.03 | 77.2 | 97.9 | 99.3 | 88.9 | |
| GPT-5.22026.03 | 77.2 | 99.3 | 98.6 | 89.1 | |
| FilteredModel=GPT-5.22026.03 | 75.4 | 96.4 | 67.9 | 79.1 | |
| VanillaModel=Claude-Sonnet-4.52026.03 | 75 | 97.9 | 97.9 | 87.5 | |
| CoTModel=Claude-Sonnet-4.52026.03 | 75 | 97.9 | 98.6 | 87.7 | |
| Claude-Sonnet-4.52026.03 | 75 | 97.9 | 97.9 | 87.5 | |
| SMSP (Ours)Model=Qwen3-VL-8B-Instruct2026.03 | 74 | 83.7 | 82.7 | 77.2 | |
| Blur with HistogramModel=GPT-5.22026.03 | 69.4 | 59.3 | 0.7 | 47.9 | |
| Blur with HistogramModel=Claude-Sonnet-4.52026.03 | 59.5 | 57.9 | 0.7 | 43 | |
| Blur with HistogramModel=Qwen3-VL-235B-A22B-Instruct2026.03 | 59.5 | 60.6 | 17.3 | 56.8 | |
| SMSP (Ours)Model=GLM-4.5V2026.03 | 58.5 | 92.9 | 88 | 69.8 | |
| SMSP (Ours)Model=Gemini-2.5-Pro2026.03 | 45.8 | 78 | 76 | 56.6 | |
| Blur with HistogramModel=Qwen3-VL-8B-Instruct2026.03 | 41.8 | 42.9 | 6.7 | 39.6 | |
| SMSP (Ours)Model=GPT-5.22026.03 | 38.4 | 72.3 | 77.3 | 50.2 | |
| Blur with HistogramModel=GLM-4.5V2026.03 | 37.1 | 64.5 | 28 | 43.8 | |
| Blur with HistogramModel=Gemini-2.5-Pro2026.03 | 36.2 | 11 | 13.3 | 27.8 | |
| FilteredModel=Gemini-2.5-Pro2026.03 | 36.1 | 77 | 74.7 | 49.8 | |
| FilteredModel=Qwen3-VL-235B-A22B-Instruct2026.03 | 33.9 | 88.3 | 93.3 | 52.7 | |
| SMSP (Ours)Model=Claude-Sonnet-4.52026.03 | 32.2 | 61 | 60 | 41.9 | |
| FilteredModel=GLM-4.5V2026.03 | 30.5 | 90.8 | 85.3 | 50.5 | |
| FilteredModel=Qwen3-VL-8B-Instruct2026.03 | 26 | 81.9 | 80 | 44.8 | |
| Blur with HistogramModel=GPT-5.22026.03 | 23.1 | 24.8 | 5.3 | 22.3 | |
| FilteredModel=GPT-5.22026.03 | 13.8 | 66.3 | 85.3 | 33 | |
| FilteredModel=Claude-Sonnet-4.52026.03 | 9.1 | 46.8 | 49.3 | 22 | |
| Blur with HistogramModel=Claude-Sonnet-4.52026.03 | 7 | 13.1 | 0 | 8.2 | |
| CoTModel=Claude-Sonnet-4.52026.03 | 5.3 | 3.9 | 9.3 | 5.2 | |
| CoTModel=GPT-5.22026.03 | 5.3 | 1.4 | 4 | 4.2 | |
| VanillaModel=Gemini-2.5-Pro2026.03 | 4.7 | 4.6 | 12 | 5.2 | |
| CoTModel=Gemini-2.5-Pro2026.03 | 4.3 | 8.5 | 13.3 | 6.1 | |
| VanillaModel=GPT-5.22026.03 | 4 | 1.4 | 4 | 3.3 | |
| VanillaModel=GLM-4.5V2026.03 | 3.6 | 6 | 13.3 | 4.9 | |
| CoTModel=GLM-4.5V2026.03 | 3.3 | 6.7 | 10.7 | 4.7 | |
| CoTModel=Qwen3-VL-235B-A22B-Instruct2026.03 | 3 | 6.4 | 9.3 | 4.4 | |
| VanillaModel=Claude-Sonnet-4.52026.03 | 3 | 3.2 | 9.3 | 3.5 | |
| VanillaModel=Qwen3-VL-235B-A22B-Instruct2026.03 | 2.7 | 7.8 | 6.7 | 4.4 | |
| VanillaModel=Qwen3-VL-8B-Instruct2026.03 | 2.4 | 6.4 | 6.7 | 3.8 | |
| CoTModel=Qwen3-VL-8B-Instruct2026.03 | 1.6 | 8.9 | 6.7 | 3.9 |