Key Information Extraction on UniKIE
77.03Bus. Trans. ScoreGemini-3-Pro
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Gemini-3-ProDeployment=On-Server2026.07 | 77.03 | 86.58 | 83.87 | 86.03 | 79.79 | 82.84 | 77.93 | 82.01 | |
| Qwen3-VL-PlusDeployment=On-Server2026.07 | 76 | 83.26 | 81.7 | 69.7 | 68.34 | 71.52 | 72.88 | 74.77 | |
| Qwen-VL-MaxDeployment=On-Server2026.07 | 74.19 | 82.51 | 76.23 | 70.35 | 67.63 | 68.71 | 73.12 | 73.25 | |
| SAYRE-4BDeployment=On-Device, Backbone=Qwen3-VL-4B, Fine-tuned=15K steps on synthetic data2026.07 | 70.38 | 80.73 | 80.97 | 78.15 | 59.97 | 64.89 | 75.36 | 72.92 | |
| Qwen3-VL-4BDeployment=On-Device2026.07 | 68.89 | 72.23 | 77.79 | 57.6 | 55.44 | 56.16 | 61.62 | 64.25 | |
| GPT-4oDeployment=On-Server2026.07 | 68.16 | 67.14 | 73.14 | 59.09 | 39.45 | 46.2 | 42.01 | 56.46 | |
| MiMo-VL-7B-RLDeployment=On-Device2026.07 | 67.91 | 74.11 | 63.22 | 64.48 | 54.72 | 54.78 | 68.22 | 63.92 | |
| SAYRE-2BDeployment=On-Device, Backbone=Qwen3-VL-2B, Fine-tuned=15K steps on synthetic data2026.07 | 67.56 | 78.35 | 78.99 | 76.1 | 56.48 | 63.13 | 72.72 | 70.47 | |
| GLM-4.1V-9BDeployment=On-Device2026.07 | 65.69 | 75.76 | 67.74 | 59.51 | 55.64 | 57.42 | 61.69 | 63.35 | |
| GPT-5Deployment=On-Server2026.07 | 64.62 | 67.37 | 71.86 | 60.12 | 41.03 | 47.57 | 40.85 | 56.2 | |
| InternVL3.5-8BDeployment=On-Device2026.07 | 64.56 | 70.54 | 68.09 | 66.86 | 40.73 | 45.19 | 44.75 | 57.25 | |
| Claude-Sonnet-4.5Deployment=On-Server2026.07 | 64.53 | 68.29 | 66.33 | 39.62 | 40.21 | 36.65 | 42.38 | 51.14 | |
| MiniCPM-V4.5-8BDeployment=On-Device2026.07 | 64.24 | 74.5 | 61.25 | 62.37 | 45.08 | 50.49 | 52.47 | 58.63 | |
| Qwen3-VL-2BDeployment=On-Device2026.07 | 64.03 | 68 | 72.98 | 59.14 | 48.77 | 50.86 | 58.57 | 60.34 | |
| Ministral-3-8BDeployment=On-Device2026.07 | 33.28 | 39.31 | 41.16 | 41.88 | 42.41 | 40.04 | 42 | 40.01 |