Measurement Information Extraction on MeasEval (test)
51.9Overall ScoreLIORI
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| LIORIsource=MeasEval Competition Top Systems2026.04 | 51.9 | 86.1 | 72.2 | 64.2 | 43.7 | 48.2 | 46.7 | 31.8 | 16.3 | 9.2 | |
| LIORIContext=Top Ranked Systems2026.04 | 51.9 | 86.1 | 72.2 | 64.2 | 43.7 | 48.2 | 46.7 | 31.8 | 16.3 | 9.2 | |
| MeasHaluprompting=Sentence-based, Model Scale=7B, GRPO=true2026.04 | 51.2 | 84.8 | 86 | 60.7 | 45.5 | 47.2 | 44.2 | 31 | 17 | 10 | |
| MeasHalu-7BModel Scale=7B, GRPO=true, Prompting=Sentence-based2026.04 | 51.2 | 84.8 | 86 | 60.7 | 45.5 | 47.2 | 44.2 | 31 | 17 | 10 | |
| MeasHaluprompting=Sentence-based, Model Scale=7B, GRPO=false2026.04 | 47.9 | 84.6 | 86.3 | 61 | 42.9 | 43.3 | 39.7 | 27.2 | 15.5 | 6.3 | |
| MeasHalu-7BModel Scale=7B, GRPO=false, Prompting=Sentence-based2026.04 | 47.9 | 84.6 | 86.3 | 61 | 42.9 | 43.3 | 39.7 | 27.2 | 15.5 | 6.3 | |
| CONNERsource=MeasEval Competition Top Systems2026.04 | 47.3 | 85.5 | 71.9 | 52.3 | 39.8 | 42.4 | 43.7 | 25.7 | 0 | 0 | |
| CONNERContext=Top Ranked Systems2026.04 | 47.3 | 85.5 | 71.9 | 52.3 | 39.8 | 42.4 | 43.7 | 25.7 | 0 | 0 | |
| MeasHaluprompting=Sentence-based, Model Scale=3B, GRPO=true2026.04 | 46 | 81 | 87.5 | 44 | 39.6 | 41.8 | 40.7 | 27.7 | 11 | 4.9 | |
| MeasHalu-3BModel Scale=3B, GRPO=true, Prompting=Sentence-based2026.04 | 46 | 81 | 87.5 | 44 | 39.6 | 41.8 | 40.7 | 27.7 | 11 | 4.9 | |
| Gemini-2.5-Proprompting=Sentence-based2026.04 | 44 | 78.2 | 88.2 | 48.6 | 43.6 | 37.6 | 38.6 | 28 | 14.3 | 5.6 | |
| Gemini-2.5-ProPrompting=Sentence-based2026.04 | 44 | 78.2 | 88.2 | 48.6 | 43.6 | 37.6 | 38.6 | 28 | 14.3 | 5.6 | |
| MeasHaluprompting=Sentence-based, Model Scale=3B, GRPO=false2026.04 | 43.3 | 78.2 | 85 | 44.9 | 37 | 37.7 | 36.5 | 24.5 | 8.4 | 4.3 | |
| MeasHalu-3BModel Scale=3B, GRPO=false, Prompting=Sentence-based2026.04 | 43.3 | 78.2 | 85 | 44.9 | 37 | 37.7 | 36.5 | 24.5 | 8.4 | 4.3 | |
| Countssource=MeasEval Competition Top Systems2026.04 | 43.2 | 86.1 | 80.4 | 61.4 | 40.6 | 31.1 | 24.5 | 18.3 | 7.7 | 6.4 | |
| CountsContext=Top Ranked Systems2026.04 | 43.2 | 86.1 | 80.4 | 61.4 | 40.6 | 31.1 | 24.5 | 18.3 | 7.7 | 6.4 | |
| GPT-5prompting=Sentence-based2026.04 | 40.6 | 72.4 | 81.7 | 50 | 39.7 | 35.1 | 35.5 | 22.6 | 13.8 | 4.2 | |
| GPT-5Prompting=Sentence-based2026.04 | 40.6 | 72.4 | 81.7 | 50 | 39.7 | 35.1 | 35.5 | 22.6 | 13.8 | 4.2 | |
| GPT-5prompting=Rule-based2026.04 | 37.1 | 80.4 | 74.2 | 39.5 | 36.1 | 27 | 35.5 | 2 | 15.2 | 5.2 | |
| GPT-5Prompting=Rule-based2026.04 | 37.1 | 80.4 | 74.2 | 39.5 | 36.1 | 27 | 35.5 | 2 | 15.2 | 5.2 | |
| Gemini-2.5-Proprompting=Rule-based2026.04 | 35.9 | 71.2 | 78.4 | 46.4 | 30.6 | 26.6 | 28.7 | 9 | 14.6 | 7.6 | |
| Gemini-2.5-ProPrompting=Rule-based2026.04 | 35.9 | 71.2 | 78.4 | 46.4 | 30.6 | 26.6 | 28.7 | 9 | 14.6 | 7.6 | |
| MeasHaluprompting=Sentence-based, Model Scale=0.5B, GRPO=true2026.04 | 34.7 | 65.9 | 76 | 26.8 | 28.4 | 27.9 | 28.7 | 18.4 | 5.9 | 3.7 | |
| MeasHalu-0.5BModel Scale=0.5B, GRPO=true, Prompting=Sentence-based2026.04 | 34.7 | 65.9 | 76 | 26.8 | 28.4 | 27.9 | 28.7 | 18.4 | 5.9 | 3.7 | |
| DeepSeek-V3prompting=Sentence-based2026.04 | 32 | 56.7 | 72.6 | 35.5 | 30.3 | 22.5 | 22.6 | 14.9 | 1.9 | 0 | |
| DeepSeek-V3Prompting=Sentence-based2026.04 | 32 | 56.7 | 72.6 | 35.5 | 30.3 | 22.5 | 22.6 | 14.9 | 1.9 | 0 | |
| MeasHaluprompting=Sentence-based, Model Scale=0.5B, GRPO=false2026.04 | 31.2 | 64.9 | 71.7 | 23.9 | 26.3 | 24.1 | 23.8 | 14 | 6.9 | 2.2 | |
| MeasHalu-0.5BModel Scale=0.5B, GRPO=false, Prompting=Sentence-based2026.04 | 31.2 | 64.9 | 71.7 | 23.9 | 26.3 | 24.1 | 23.8 | 14 | 6.9 | 2.2 | |
| DeepSeek-R1prompting=Sentence-based2026.04 | 30.4 | 58.9 | 71.1 | 35.6 | 26 | 19.8 | 18.2 | 11.8 | 11.3 | 5.7 | |
| DeepSeek-R1Prompting=Sentence-based2026.04 | 30.4 | 58.9 | 71.1 | 35.6 | 26 | 19.8 | 18.2 | 11.8 | 11.3 | 5.7 | |
| Qwen2.5-72b-instprompting=Rule-based2026.04 | 28.6 | 64.4 | 82.6 | 23.6 | 19.6 | 14.7 | 16.4 | 0.1 | 7.6 | 2.1 | |
| Qwen2.5-72b-instPrompting=Rule-based2026.04 | 28.6 | 64.4 | 82.6 | 23.6 | 19.6 | 14.7 | 16.4 | 0.1 | 7.6 | 2.1 | |
| DeepSeek-V3prompting=Rule-based2026.04 | 27.1 | 65.7 | 76.8 | 21.4 | 23.9 | 13.5 | 11.3 | 0.1 | 8.5 | 1.4 | |
| DeepSeek-V3Prompting=Rule-based2026.04 | 27.1 | 65.7 | 76.8 | 21.4 | 23.9 | 13.5 | 11.3 | 0.1 | 8.5 | 1.4 | |
| DeepSeek-R1prompting=Rule-based2026.04 | 25.3 | 56.9 | 58.6 | 24 | 21.6 | 16.3 | 16.3 | 2.4 | 8.5 | 2.9 | |
| DeepSeek-R1Prompting=Rule-based2026.04 | 25.3 | 56.9 | 58.6 | 24 | 21.6 | 16.3 | 16.3 | 2.4 | 8.5 | 2.9 | |
| Baselinesource=MeasEval Competition Top Systems2026.04 | 23.9 | 82.7 | 56.1 | 0 | 5.3 | 7.5 | 6.4 | 0.7 | 0.5 | 0 | |
| BaselineContext=Top Ranked Systems2026.04 | 23.9 | 82.7 | 56.1 | 0 | 5.3 | 7.5 | 6.4 | 0.7 | 0.5 | 0 | |
| Qwen2.5-72b-instprompting=Sentence-based2026.04 | 21.2 | 40.3 | 51.6 | 23.2 | 20.4 | 13.7 | 11.3 | 8.7 | 3.8 | 1.2 | |
| Qwen2.5-72b-instPrompting=Sentence-based2026.04 | 21.2 | 40.3 | 51.6 | 23.2 | 20.4 | 13.7 | 11.3 | 8.7 | 3.8 | 1.2 | |
| Qwen2.5-7b-instprompting=Rule-based2026.04 | 17.1 | 49.1 | 47.8 | 10.6 | 8.8 | 4.5 | 5.7 | 0 | 4 | 1.7 | |
| Qwen2.5-7b-instPrompting=Rule-based2026.04 | 17.1 | 49.1 | 47.8 | 10.6 | 8.8 | 4.5 | 5.7 | 0 | 4 | 1.7 | |
| Qwen2.5-7b-instprompting=Sentence-based2026.04 | 7.3 | 15.1 | 16 | 2.7 | 6.6 | 5.9 | 4.4 | 3.1 | 0.3 | 0.5 | |
| Qwen2.5-7b-instPrompting=Sentence-based2026.04 | 7.3 | 15.1 | 16 | 2.7 | 6.6 | 5.9 | 4.4 | 3.1 | 0.3 | 0.5 |