Information Extraction on 1,000-note gold-standard set 1.0 (test)
94.8Micro PrecisionQwen2.5-14B-Instruct
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Qwen2.5-14B-InstructPost-training Setting=DPO2026.05 | 94.8 | 79.4 | 86.4 | 94.8 | 76.9 | 83.7 | |
| Llama-3.1-8B-InstructPost-training Setting=DPO2026.05 | 94.8 | 70.1 | 80.6 | 94.9 | 69.6 | 79.7 | |
| Qwen2.5-7B-InstructPost-training Setting=DPO2026.05 | 92.5 | 70.5 | 80 | 90.8 | 68.3 | 76.9 | |
| gemma-2-9b-itPost-training Setting=DPO2026.05 | 91 | 1 | 2 | 62.6 | 2.1 | 3.5 | |
| OLMo-2-1124-7B-DPOPost-training Setting=DPO2026.05 | 79.9 | 64.6 | 71.4 | 78.8 | 58.4 | 64.6 | |
| Qwen2.5-14B-InstructPost-training Setting=Base2026.05 | 79.8 | 76.2 | 78 | 79.1 | 74.8 | 75.5 | |
| Qwen2.5-7B-InstructPost-training Setting=Base2026.05 | 73.6 | 71 | 72.3 | 74.2 | 71.9 | 71.5 | |
| Llama-3.1-8B-InstructPost-training Setting=SFT2026.05 | 69.4 | 83.8 | 75.9 | 69.3 | 85.3 | 73.8 | |
| granite-3.1-8b-instructPost-training Setting=Base2026.05 | 64.6 | 63.9 | 64.3 | 69 | 66.3 | 66.1 | |
| OLMo-2-1124-7B-DPOPost-training Setting=SFT2026.05 | 63.1 | 70.6 | 66.6 | 57.4 | 58.3 | 55.4 | |
| OLMo-2-1124-7B-DPOPost-training Setting=Base2026.05 | 61.7 | 32.9 | 42.9 | 54.6 | 28.6 | 35.1 | |
| Qwen2.5-14B-InstructPost-training Setting=SFT2026.05 | 61.6 | 89.5 | 73 | 62.3 | 88.1 | 70.9 | |
| gemma-2-9b-itPost-training Setting=SFT2026.05 | 57.5 | 63.7 | 60.4 | 65 | 64.9 | 62.2 | |
| Qwen2.5-7B-InstructPost-training Setting=SFT2026.05 | 57.1 | 82.3 | 67.4 | 61 | 83.3 | 66.9 | |
| OLMo-2-1124-13B-DPOPost-training Setting=Base2026.05 | 56.5 | 72.5 | 63.5 | 65.2 | 69.6 | 64.6 | |
| Llama-3.1-8B-InstructPost-training Setting=Base2026.05 | 52.3 | 76.1 | 62 | 56.3 | 78.3 | 61.5 | |
| gemma-2-9b-itPost-training Setting=Base2026.05 | 51 | 80.3 | 62.4 | 58.8 | 75.9 | 63 | |
| Llama-3.2-3B-InstructPost-training Setting=SFT2026.05 | 44.9 | 59.2 | 51.1 | 42.2 | 51.3 | 42.4 | |
| OLMo-2-1124-13B-DPOPost-training Setting=SFT2026.05 | 43.3 | 86.3 | 57.7 | 46.3 | 84.2 | 57.2 | |
| Mistral-Small-Instruct-2409Post-training Setting=SFT2026.05 | 40.5 | 62.8 | 49.2 | 42.4 | 71.7 | 47 | |
| granite-3.1-8b-instructPost-training Setting=SFT2026.05 | 38.2 | 89.2 | 53.5 | 43.9 | 89.2 | 54.7 | |
| Mistral-Small-Instruct-2409Post-training Setting=Base2026.05 | 31.8 | 43 | 36.5 | 44.8 | 45.8 | 37 | |
| Mistral-7B-Instruct-v0.3Post-training Setting=Base2026.05 | 29.6 | 37.8 | 33.2 | 36.1 | 29.6 | 26.7 | |
| Llama-3.2-3B-InstructPost-training Setting=Base2026.05 | 25.1 | 46.6 | 32.6 | 36.1 | 46.7 | 35.3 | |
| Mistral-7B-Instruct-v0.3Post-training Setting=SFT2026.05 | 15.8 | 86.8 | 26.7 | 21.6 | 84.6 | 30.7 |