Document Text Transcription on Malvern-Hills-5+
5.43CEROCR+PAGE1
Evaluation Results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| OCR+PAGE1Base Model=GEMINI2025.02 | 5.43 | 31.15 | 5.83 | 4.32 | 21 | 0.77 | 2.68 | 2.1 | 21.4 | 0 | |
| IMAGESBase Model=GEMINI2025.02 | 5.63 | 27.37 | 0 | 7.41 | 19.96 | 4.52 | 1.41 | 2 | 30.5 | 0 | |
| OCR+IMAGESBase Model=GEMINI2025.02 | 5.69 | 35.75 | 5.83 | 10.24 | 19.67 | 4.52 | 3.68 | 1.97 | 27.5 | 0 | |
| OCR+PAGENBase Model=GEMINI2025.02 | 5.86 | 31.43 | 5.83 | 4.55 | 21.05 | 0.77 | 2.7 | 2.1 | 31.2 | 0 | |
| OCR+PAGERBase Model=GEMINI2025.02 | 5.99 | 31.22 | 5.83 | 4.35 | 21.04 | 0.77 | 2.7 | 2.1 | 20.9 | 0 | |
| IMAGES-AAOBase Model=GEMINI2025.02 | 6.14 | 27.12 | 0 | 5.97 | 21.15 | 4.52 | 0.26 | 2.12 | 19.2 | 0 | |
| OCR+IMAGESBase Model=GPT-4O2025.02 | 6.49 | 44.21 | 5.83 | 20.25 | 18.13 | 4.53 | 3.57 | 1.81 | 83.1 | 0 | |
| OCRBase Model=GEMINI2025.02 | 7.05 | 29.46 | 5.83 | 4.2 | 19.43 | 0 | 3.36 | 1.94 | 24 | 0 | |
| OCR+PAGENBase Model=GPT-4O2025.02 | 7.59 | 33.9 | 5.83 | 8.81 | 19.26 | 0.77 | 2.53 | 1.92 | 68.2 | 0 | |
| OCR+PAGE1Base Model=GPT-4O2025.02 | 9.12 | 33.32 | 5.83 | 8.55 | 18.94 | 0.77 | 2.51 | 1.89 | 51.8 | 0 | |
| OCRBase Model=GPT-4O2025.02 | 9.42 | 33.7 | 5.83 | 8.35 | 19.52 | 0.07 | 3.27 | 1.95 | 47.1 | 0 | |
| OCR+IMAGESBase Model=GEMMA2025.02 | 9.84 | 7.4 | 5.83 | 0.57 | 0.99 | 4.52 | 3.68 | 1.99 | 44.8 | 0 | |
| OCR+PAGERBase Model=GPT-4O2025.02 | 10.17 | 34.15 | 5.83 | 8.61 | 19.71 | 0.77 | 2.53 | 1.97 | 46.6 | 0 | |
| IMAGESBase Model=GPT-4O2025.02 | 10.24 | 33.84 | 0 | 14.77 | 19.06 | 4.46 | 1.38 | 1.91 | 90.8 | 0 | |
| IMAGES-AAOBase Model=GPT-4O2025.02 | 12.38 | 31.02 | 0 | 12.25 | 18.77 | 4.46 | 0.25 | 1.88 | 80.2 | 0 | |
| OCRBase Model=GEMMA2025.02 | 12.92 | 7.1 | 5.83 | 0.24 | 1.03 | 0 | 3.36 | 2.06 | 44 | 0 | |
| OCRInput Strategy=OCR baseline2025.02 | 13.96 | 5.83 | 5.83 | 0 | 0 | 0 | 0 | 0 | — | 0 | |
| IMAGESBase Model=GEMMA2025.02 | 16 | 1.43 | 0 | 0.41 | 1.01 | 4.52 | 1.41 | 2.03 | 85.4 | 0.58 | |
| IMAGES-AAOBase Model=GEMMA2025.02 | 17.64 | 1.32 | 0 | 0.33 | 0.99 | 4.52 | 0.26 | 1.98 | 92.4 | 0.75 | |
| OCR+PAGE1Base Model=GEMMA2025.02 | 21.35 | 7.03 | 5.83 | 0.24 | 0.95 | 0.77 | 2.68 | 1.9 | 89.8 | 0.75 | |
| OCR+PAGERBase Model=GEMMA2025.02 | 23.65 | 7.06 | 5.83 | 0.24 | 0.98 | 0.77 | 2.7 | 1.97 | 79.4 | 0.58 | |
| OCR+PAGENBase Model=GEMMA2025.02 | 24.94 | 7.22 | 5.83 | 0.44 | 0.95 | 0.77 | 2.7 | 1.81 | 126 | 1.12 | |
| TrOCRInput Strategy=Lines, Zero-shot=true2025.02 | 31.43 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 990 | 0 | |
| DOCOWL2Input Strategy=Full Page, Zero-shot=true2025.02 | 92.08 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 234 | 0 | |
| DOCOWL2-LINESInput Strategy=Lines, Zero-shot=true2025.02 | 93.01 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2,054 | 0 |