OCR Verification on Curated dataset of 1,000 PDF pages 1.0 (test)
72.27F1 ScoreVLM-as-Judge
Evaluation results
| Method | Links | |
|---|---|---|
| VLM-as-JudgeReference VLM=Qwen2-VL-72B, Human Score Band=0.9-1.02025.04 | 72.27 | |
| VLM-as-JudgeReference VLM=GPT4o, Human Score Band=0.9-1.02025.04 | 72.12 | |
| CEReference VLM=Qwen2-VL-7B, Human Score Band=0.9-1.02025.04 | 70.88 | |
| VLM-as-JudgeReference VLM=Qwen2-VL-7B, Human Score Band=0.9-1.02025.04 | 70.05 | |
| CEReference VLM=Qwen2-VL-72B, Human Score Band=0.0-0.32025.04 | 69.57 | |
| CEReference VLM=Qwen2-VL-7B, Human Score Band=0.0-0.32025.04 | 67.42 | |
| CEReference VLM=Qwen2-VL-72B, Human Score Band=0.9-1.02025.04 | 66.13 | |
| CEReference VLM=GPT4o, Human Score Band=0.0-0.32025.04 | 65.44 | |
| CEReference VLM=GPT4o, Human Score Band=0.9-1.02025.04 | 57.29 | |
| VLM-as-JudgeReference VLM=Qwen2-VL-7B, Human Score Band=0.0-0.32025.04 | 54.74 | |
| VLM-as-JudgeReference VLM=Qwen2-VL-72B, Human Score Band=0.0-0.32025.04 | 52.83 | |
| CEReference VLM=Qwen2-VL-7B, Human Score Band=Overall2025.04 | 51.3 | |
| CEReference VLM=Qwen2-VL-72B, Human Score Band=Overall2025.04 | 51 | |
| VLM-as-JudgeReference VLM=GPT4o, Human Score Band=0.0-0.32025.04 | 48.21 | |
| CEReference VLM=GPT4o, Human Score Band=Overall2025.04 | 48 | |
| CEReference VLM=GPT4o, Human Score Band=0.7-0.82025.04 | 40.19 | |
| CEReference VLM=Qwen2-VL-72B, Human Score Band=0.7-0.82025.04 | 40.14 | |
| VLM-as-JudgeReference VLM=GPT4o, Human Score Band=Overall2025.04 | 40 | |
| VLM-as-JudgeReference VLM=Qwen2-VL-72B, Human Score Band=Overall2025.04 | 39.8 | |
| CEReference VLM=Qwen2-VL-7B, Human Score Band=0.7-0.82025.04 | 39.56 | |
| CEReference VLM=GPT4o, Human Score Band=0.4-0.62025.04 | 38.67 | |
| VLM-as-JudgeReference VLM=Qwen2-VL-7B, Human Score Band=Overall2025.04 | 36.1 | |
| CEReference VLM=Qwen2-VL-72B, Human Score Band=0.4-0.62025.04 | 28.47 | |
| CEReference VLM=Qwen2-VL-7B, Human Score Band=0.4-0.62025.04 | 27.34 | |
| VLM-as-JudgeReference VLM=Qwen2-VL-72B, Human Score Band=0.7-0.82025.04 | 23.01 | |
| VLM-as-JudgeReference VLM=GPT4o, Human Score Band=0.4-0.62025.04 | 21.47 | |
| VLM-as-JudgeReference VLM=Qwen2-VL-7B, Human Score Band=0.7-0.82025.04 | 19.82 | |
| VLM-as-JudgeReference VLM=GPT4o, Human Score Band=0.7-0.82025.04 | 18.42 | |
| VLM-as-JudgeReference VLM=Qwen2-VL-72B, Human Score Band=0.4-0.62025.04 | 11.43 | |
| VLM-as-JudgeReference VLM=Qwen2-VL-7B, Human Score Band=0.4-0.62025.04 | 0 |