Transcription on Malvern-Hills dataset (test)
5.83CER (%)GEMINI-2.5-PRO
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GEMINI-2.5-PROInput=OCR+PAGE12025.02 | 5.83 | 13.24 | |
| GEMINI-2.5-PROInput=IMAGES2025.02 | 6.42 | 10.88 | |
| GEMINI-2.5-PROInput=OCR+IMAGES2025.02 | 6.46 | 14.18 | |
| GEMINI-2.5-PROInput=OCR+PAGEN2025.02 | 6.54 | 14.41 | |
| GPT-4OInput=OCR+IMAGES2025.02 | 7.11 | 17.54 | |
| GEMINI-2.5-PROInput=OCR2025.02 | 7.47 | 11.67 | |
| GPT-4OInput=OCR+PAGEN2025.02 | 8.05 | 15.15 | |
| GEMINI-2.5-PROInput=IMAGES-ALL-AT-ONCE2025.02 | 8.2 | 11.01 | |
| GPT-4OInput=OCR+PAGE12025.02 | 8.92 | 15 | |
| GEMMA-3-27BInput=OCR+IMAGES2025.02 | 10.54 | 2.84 | |
| GPT-4OInput=OCR2025.02 | 10.6 | 13.32 | |
| GEMMA-3-27BInput=OCR+PAGEN2025.02 | 11.22 | 3.01 | |
| GPT-4OInput=IMAGES2025.02 | 11.24 | 13.46 | |
| GPT-4OInput=IMAGES-ALL-AT-ONCE2025.02 | 11.35 | 13 | |
| GEMMA-3-27BInput=OCR+PAGE12025.02 | 12.55 | 2.87 | |
| GEMMA-3-27BInput=OCR2025.02 | 13.52 | 2.8 | |
| Azure Vision OCRInput=OCR2025.02 | 14.41 | 2.3 | |
| GEMMA-3-27BInput=IMAGES-ALL-AT-ONCE2025.02 | 15.21 | 0.6 | |
| GEMMA-3-27BInput=IMAGES2025.02 | 27.19 | 0.66 |