Note-table consistency verification on EHRCon Discharge notes
77.36RecallEHR-INSPECTOR
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| EHR-INSPECTORBase LLM=Gemini 2.5 Flash, Evaluation Setting=Lenient2026.05 | 77.36 | — | — | — | — | 71.67 | |
| EHR-INSPECTORBase LLM=Gemini 2.5 Flash, Evaluation Setting=Harsh2026.05 | 69.28 | — | — | — | — | 62.81 | |
| EHR-INSPECTORBase LLM=Qwen3 32B, Evaluation Setting=Lenient2026.05 | 56.18 | — | — | — | — | 65.29 | |
| EHR-INSPECTORBase LLM=Qwen3 32B, Evaluation Setting=Harsh2026.05 | 50.89 | — | — | — | — | 56.12 | |
| CheckEHRBase LLM=Gemini 2.5 Flash, Evaluation Setting=Lenient2026.05 | 48.8 | — | — | — | — | 65.14 | |
| EHR-INSPECTORBase LLM=MedGemma 27B, Evaluation Setting=Lenient2026.05 | 48.68 | — | — | — | — | 44.99 | |
| CheckEHRBase LLM=Gemini 2.5 Flash, Evaluation Setting=Harsh2026.05 | 44.89 | — | — | — | — | 55.01 | |
| EHR-INSPECTORBase LLM=MedGemma 27B, Evaluation Setting=Harsh2026.05 | 42.74 | — | — | — | — | 39.51 | |
| EHR-INSPECTORBase LLM=GPT-OSS 20B, Evaluation Setting=Lenient2026.05 | 30 | — | — | — | — | 59.12 | |
| CheckEHRBase LLM=GPT-OSS 20B, Evaluation Setting=Lenient2026.05 | 29.4 | — | — | — | — | 60.19 | |
| EHR-INSPECTORBase LLM=GPT-OSS 20B, Evaluation Setting=Harsh2026.05 | 26.99 | — | — | — | — | 52.45 | |
| CheckEHRBase LLM=GPT-OSS 20B, Evaluation Setting=Harsh2026.05 | 26.5 | — | — | — | — | 50.7 | |
| CheckEHRBase LLM=Qwen3 32B, Evaluation Setting=Lenient2026.05 | 26.05 | — | — | — | — | 62.57 | |
| CheckEHRBase LLM=Qwen3 32B, Evaluation Setting=Harsh2026.05 | 23.04 | — | — | — | — | 54.02 | |
| CheckEHRBase LLM=MedGemma 27B, Evaluation Setting=Lenient2026.05 | 17.95 | — | — | — | — | 55.19 | |
| CheckEHRBase LLM=MedGemma 27B, Evaluation Setting=Harsh2026.05 | 16.41 | — | — | — | — | 43.23 | |
| CheckEHR2026.05 | — | 69.79 | 61.65 | 68.72 | 55.74 | — | |
| EHR-INSPECTOR2026.05 | — | 83.85 | 91.72 | 75.36 | 78.93 | — |