# Claim Extraction on MMCE Intent-Critical

Source: [Claim Extraction on MMCE Intent-Critical](<https://www.sota2.com/research/sota/claim-extraction-on-mmce-intent-critical>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-21T07:14:36.668Z

Evaluation of Claim Extraction performance on the MMCE Intent-Critical dataset, measured using Reference-Based, Entailment (Strict/Lenient), and Decontextualization (Strict/Lenient) metrics.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Claim Extraction |
| Dataset (including any stated split) | MMCE Intent-Critical |
| Metric | Reference-Based Score |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 4 of 4 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Reference-Based Score](<https://www.sota2.com/research/sota/claim-extraction-on-mmce-intent-critical.md?metric=Reference-Based%20Score>): Higher is better; unit: SCALAR.

- [Entailment (Strict) Accuracy](<https://www.sota2.com/research/sota/claim-extraction-on-mmce-intent-critical.md?metric=Entailment%20(Strict)%20Accuracy>): Higher is better; unit: PERCENT.

- [Entailment (Lenient) Accuracy](<https://www.sota2.com/research/sota/claim-extraction-on-mmce-intent-critical.md?metric=Entailment%20(Lenient)%20Accuracy>): Higher is better; unit: PERCENT.

- [Decontextualization (Strict) Accuracy](<https://www.sota2.com/research/sota/claim-extraction-on-mmce-intent-critical.md?metric=Decontextualization%20(Strict)%20Accuracy>): Higher is better; unit: PERCENT.

- [Decontextualization (Lenient) Accuracy](<https://www.sota2.com/research/sota/claim-extraction-on-mmce-intent-critical.md?metric=Decontextualization%20(Lenient)%20Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Reference-Based Score | Result date | Report paper |
| --- | --- | --- | --- |
| MICE | 3.56 | Not recorded | 69e704827d0172140ec7ec10 |
| MICE | 3.31 | Not recorded | 69e704827d0172140ec7ec10 |
| MLLM with ICL | 2.6 | Not recorded | 69e704827d0172140ec7ec10 |
| MLLM with ICL | 2.08 | Not recorded | 69e704827d0172140ec7ec10 |

### MICE

Reference-Based Score: 3.56. Result ID: 69e7228947f34ce924ccdb83.

[Reporting paper](<https://www.sota2.com/research/paper/69e704827d0172140ec7ec10.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69e704827d0172140ec7ec10.md>)

Recorded evaluation configuration: {"Model":"Gemini 2.0 Flash","Evaluation Protocol":"LLM-based Evaluation"}. This record does not assert independent reproduction.

### MICE

Reference-Based Score: 3.31. Result ID: 69e7228947f34ce924ccdb81.

[Reporting paper](<https://www.sota2.com/research/paper/69e704827d0172140ec7ec10.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69e704827d0172140ec7ec10.md>)

Recorded evaluation configuration: {"Model":"Gemini 2.0 Flash","Evaluation Protocol":"Human Evaluation"}. This record does not assert independent reproduction.

### MLLM with ICL

Reference-Based Score: 2.6. Result ID: 69e7228947f34ce924ccdb80.

[Reporting paper](<https://www.sota2.com/research/paper/69e704827d0172140ec7ec10.md>)

Recorded evaluation configuration: {"Model":"Gemini 2.0 Flash","Evaluation Protocol":"Human Evaluation"}. This record does not assert independent reproduction.

### MLLM with ICL

Reference-Based Score: 2.08. Result ID: 69e7228947f34ce924ccdb82.

[Reporting paper](<https://www.sota2.com/research/paper/69e704827d0172140ec7ec10.md>)

Recorded evaluation configuration: {"Model":"Gemini 2.0 Flash","Evaluation Protocol":"LLM-based Evaluation"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
