Bank Statement Question Answering on MortarBench 1.0 (test)
83.6Overall F1Gemini 3.1 Pro + CRIT
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.1 Pro + CRITBackbone Model=Gemini 3.1 Pro, CRIT Agent Integration=true2026.06 | 83.6 | 80.5 | 97 | 77 | 96.7 | 29.3 | 1 | 3.3 | 6 | |
| Gemini 3.1 ProBackbone Model=Gemini 3.1 Pro, CRIT Agent Integration=false2026.06 | 80.8 | 77.1 | 97 | 72.2 | 96.7 | 36.7 | 1.1 | 0 | 0 | |
| GPT-5.5 + CRITBackbone Model=GPT-5.5, CRIT Agent Integration=true2026.06 | 80.2 | 77.3 | 93.5 | 73.1 | 95 | 31 | 1.2 | 7.7 | 14 | |
| GPT-5.5Backbone Model=GPT-5.5, CRIT Agent Integration=false2026.06 | 79.9 | 76.8 | 94.6 | 72.3 | 93.3 | 34.7 | 1.9 | 1.3 | 3 | |
| Claude Sonnet 4.6 + CRITBackbone Model=Claude Sonnet 4.6, CRIT Agent Integration=true2026.06 | 62.3 | 59.4 | 36.9 | 72.6 | 90 | 34.3 | 1.2 | 2.7 | 5 | |
| Claude Sonnet 4.6Backbone Model=Claude Sonnet 4.6, CRIT Agent Integration=false2026.06 | 56.1 | 51.4 | 36.3 | 63 | 85 | 48.7 | 2.5 | 1.7 | 5 |