# Detection of quotation errors on SCIFACT Full

Source: [Detection of quotation errors on SCIFACT Full](<https://www.sota2.com/research/sota/detection-of-quotation-errors-on-scifact-full>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-09T09:03:44.177Z

Evaluation of model performance on the full SCIFACT dataset specifically targeting the detection of quotation errors, measured using SUP F1, CON F1, NUL F1, and Macro-Average F1.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Detection of quotation errors |
| Dataset (including any stated split) | SCIFACT Full |
| Metric | SUP F1 |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [SUP F1](<https://www.sota2.com/research/sota/detection-of-quotation-errors-on-scifact-full.md?metric=SUP%20F1>): Higher is better; unit: PERCENT.

- [CON F1](<https://www.sota2.com/research/sota/detection-of-quotation-errors-on-scifact-full.md?metric=CON%20F1>): Higher is better; unit: PERCENT.

- [NUL F1](<https://www.sota2.com/research/sota/detection-of-quotation-errors-on-scifact-full.md?metric=NUL%20F1>): Higher is better; unit: PERCENT.

- [Macro-Avg F1](<https://www.sota2.com/research/sota/detection-of-quotation-errors-on-scifact-full.md?metric=Macro-Avg%20F1>): Higher is better; unit: PERCENT.

## Recorded results

| Method | SUP F1 | Result date | Report paper |
| --- | --- | --- | --- |
| Q7B(FT) | 88.19 | Not recorded | 6a279e2490966ade919bf7a2 |
| DV3 | 87.6 | Not recorded | 6a279e2490966ade919bf7a2 |
| QMax | 87.11 | Not recorded | 6a279e2490966ade919bf7a2 |
| QMax | 86.56 | Not recorded | 6a279e2490966ade919bf7a2 |
| DV3 | 85.56 | Not recorded | 6a279e2490966ade919bf7a2 |
| DR1 | 83.91 | Not recorded | 6a279e2490966ade919bf7a2 |
| QPlus | 82.53 | Not recorded | 6a279e2490966ade919bf7a2 |
| QPlus | 82.19 | Not recorded | 6a279e2490966ade919bf7a2 |
| Rag(hybird) | 81.44 | Not recorded | 6a279e2490966ade919bf7a2 |
| Rag(hyde) | 79.07 | Not recorded | 6a279e2490966ade919bf7a2 |
| D-L8B | 77.35 | Not recorded | 6a279e2490966ade919bf7a2 |
| DR1 | 64.93 | Not recorded | 6a279e2490966ade919bf7a2 |
| D-L8(FT) | 63.02 | Not recorded | 6a279e2490966ade919bf7a2 |
| Q7B | 60.98 | Not recorded | 6a279e2490966ade919bf7a2 |

### Q7B(FT)

SUP F1: 88.19. Result ID: 6a27d67efbc373592c24c297.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

Recorded evaluation configuration: {"Strategy":"Fine-tuned"}. This record does not assert independent reproduction.

### DV3

SUP F1: 87.6. Result ID: 6a27d67efbc373592c24c28b.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77001e209131cea7066a.md>)

Recorded evaluation configuration: {"Strategy":"Zero-shot"}. This record does not assert independent reproduction.

### QMax

SUP F1: 87.11. Result ID: 6a27d67efbc373592c24c28e.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

Recorded evaluation configuration: {"Strategy":"Zero-shot"}. This record does not assert independent reproduction.

### QMax

SUP F1: 86.56. Result ID: 6a27d67efbc373592c24c292.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

Recorded evaluation configuration: {"Strategy":"3-shot"}. This record does not assert independent reproduction.

### DV3

SUP F1: 85.56. Result ID: 6a27d67efbc373592c24c28f.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77001e209131cea7066a.md>)

Recorded evaluation configuration: {"Strategy":"3-shot"}. This record does not assert independent reproduction.

### DR1

SUP F1: 83.91. Result ID: 6a27d67efbc373592c24c28c.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Strategy":"Zero-shot"}. This record does not assert independent reproduction.

### QPlus

SUP F1: 82.53. Result ID: 6a27d67efbc373592c24c291.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

Recorded evaluation configuration: {"Strategy":"3-shot"}. This record does not assert independent reproduction.

### QPlus

SUP F1: 82.19. Result ID: 6a27d67efbc373592c24c28d.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

Recorded evaluation configuration: {"Strategy":"Zero-shot"}. This record does not assert independent reproduction.

### Rag(hybird)

SUP F1: 81.44. Result ID: 6a27d67efbc373592c24c293.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

Recorded evaluation configuration: {"Strategy":"RAG"}. This record does not assert independent reproduction.

### Rag(hyde)

SUP F1: 79.07. Result ID: 6a27d67efbc373592c24c294.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e3b1e209131cea3e835.md>)

[Implementation](<https://github.com/texttron/hyde>)

Recorded evaluation configuration: {"Strategy":"RAG (HyDE)"}. This record does not assert independent reproduction.

### D-L8B

SUP F1: 77.35. Result ID: 6a27d67efbc373592c24c296.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

Recorded evaluation configuration: {"Strategy":"Zero-shot"}. This record does not assert independent reproduction.

### DR1

SUP F1: 64.93. Result ID: 6a27d67efbc373592c24c290.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Strategy":"3-shot"}. This record does not assert independent reproduction.

### D-L8(FT)

SUP F1: 63.02. Result ID: 6a27d67efbc373592c24c298.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

Recorded evaluation configuration: {"Strategy":"Fine-tuned"}. This record does not assert independent reproduction.

### Q7B

SUP F1: 60.98. Result ID: 6a27d67efbc373592c24c295.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e2490966ade919bf7a2.md>)

Recorded evaluation configuration: {"Strategy":"Zero-shot"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
