Measurement Information Extraction on MeasEval-Ext (test)
57.8Overall ScoreMeasHalu-7B
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| MeasHalu-7BAlignment=Quantitative domain alignment2026.04 | 57.8 | 86.1 | 83.2 | 53.9 | 55.5 | 55.1 | 52.2 | 45.9 | 15.9 | 9.7 | |
| MeasHalu-7BModel Scale=7B, Prompting=Sentence-based2026.04 | 57.8 | 86.1 | 83.2 | 53.9 | 55.5 | 55.1 | 52.2 | 45.9 | 15.9 | 9.7 | |
| Gemini-2.5-ProPrompting Strategy=Sentence-based Prompting2026.04 | 46.2 | 82.7 | 83.2 | 44.4 | 47.2 | 39.9 | 40.2 | 33.2 | 10 | 7.2 | |
| Gemini-2.5-ProPrompting=Sentence-based2026.04 | 46.2 | 82.7 | 83.2 | 44.4 | 47.2 | 39.9 | 40.2 | 33.2 | 10 | 7.2 | |
| GPT-5Prompting Strategy=Sentence-based Prompting2026.04 | 40.2 | 75 | 75.9 | 45.8 | 43.5 | 30.3 | 30.3 | 23.9 | 10 | 7 | |
| Gemini-2.5-ProPrompting Strategy=Rule-based Prompting2026.04 | 38.6 | 76.6 | 70.7 | 44.4 | 35.1 | 31.2 | 29.1 | 15.1 | 14 | 5.5 | |
| GPT-5Prompting Strategy=Rule-based Prompting2026.04 | 38.3 | 83.3 | 70.6 | 35.7 | 40 | 28.2 | 31 | 4.6 | 13.5 | 1.9 | |
| GPT-5Prompting=Rule-based2026.04 | 38.3 | 83.3 | 70.6 | 35.7 | 40 | 28.2 | 31 | 4.6 | 13.5 | 1.9 | |
| DeepSeek-R1Prompting Strategy=Sentence-based Prompting2026.04 | 32.4 | 62.2 | 64.8 | 25.5 | 29.9 | 23.5 | 21.5 | 14.6 | 7.5 | 6.9 | |
| DeepSeek-V3Prompting Strategy=Rule-based Prompting2026.04 | 31.2 | 72.4 | 72.6 | 23.4 | 27 | 18.9 | 12.1 | 1.2 | 11.3 | 3.7 | |
| DeepSeek-V3Prompting Strategy=Sentence-based Prompting2026.04 | 29.9 | 52.1 | 56.5 | 22.9 | 29.6 | 22.9 | 20.9 | 15.5 | 5.3 | 4.8 | |
| Qwen2.5-72bPrompting Strategy=Rule-based Prompting2026.04 | 29.6 | 67.5 | 74.7 | 20.3 | 24.6 | 18.7 | 14.2 | 0 | 9.1 | 6.6 | |
| DeepSeek-R1Prompting Strategy=Rule-based Prompting2026.04 | 25.2 | 55.3 | 51.4 | 21.7 | 21.3 | 16.9 | 14.1 | 4.5 | 9.9 | 5.2 | |
| Qwen2.5-72bPrompting Strategy=Sentence-based Prompting2026.04 | 20.2 | 34.3 | 38.3 | 18.3 | 20.7 | 14.5 | 12.7 | 10.6 | 4.8 | 5 | |
| Qwen2.5-7bPrompting Strategy=Rule-based Prompting2026.04 | 18.1 | 50.1 | 35.3 | 14.8 | 10.8 | 7.8 | 6.9 | 0.1 | 3.8 | 0.6 | |
| Qwen2.5-7bPrompting Strategy=Sentence-based Prompting2026.04 | 3.3 | 6.5 | 5.6 | 2.2 | 3 | 2.8 | 1.7 | 0.8 | 1.6 | 2 |