# PII detection on PUPA

Source: [PII detection on PUPA](<https://www.sota2.com/research/sota/pii-detection-on-pupa>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-13T09:57:00.544Z

Evaluation of Personally Identifiable Information (PII) detection performance on the PUPA dataset, measured using the F1 score.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | PII detection |
| Dataset (including any stated split) | PUPA |
| Metric | F1 Score |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 8 of 8 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [F1 Score](<https://www.sota2.com/research/sota/pii-detection-on-pupa.md?metric=F1%20Score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | F1 Score | Result date | Report paper |
| --- | --- | --- | --- |
| Fine-tuning | 68.48 | Not recorded | 69dc78a17d0172140ec78beb |
| DSPy MIPROv2 | 64.91 | Not recorded | 69dc78a17d0172140ec78beb |
| AIR | 59.32 | Not recorded | 69dc78a17d0172140ec78beb |
| DSPy GEPA | 59.29 | Not recorded | 69dc78a17d0172140ec78beb |
| TextGrad | 55.54 | Not recorded | 69dc78a17d0172140ec78beb |
| DSPy BootstrapFewShot (#3) | 44.23 | Not recorded | 69dc78a17d0172140ec78beb |
| KNN | 35.08 | Not recorded | 69dc78a17d0172140ec78beb |
| Initial prompt | 17.58 | Not recorded | 69dc78a17d0172140ec78beb |

### Fine-tuning

F1 Score: 68.48. Result ID: 69dcbdd547f34ce924c42033.

[Reporting paper](<https://www.sota2.com/research/paper/69dc78a17d0172140ec78beb.md>)

Recorded evaluation configuration: {"Train Tokens (Base Model)":"190881 / 7086","Inference Tokens (Base Model)":"19263 / 611"}. This record does not assert independent reproduction.

### DSPy MIPROv2

F1 Score: 64.91. Result ID: 69dcbdd547f34ce924c42035.

[Reporting paper](<https://www.sota2.com/research/paper/69dc78a17d0172140ec78beb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75121e209131cea65828.md>)

Recorded evaluation configuration: {"Train Tokens (Base Model)":"1490233 / 134750","Inference Tokens (Base Model)":"52409 / 1480"}. This record does not assert independent reproduction.

### AIR

F1 Score: 59.32. Result ID: 69dcbdd547f34ce924c42038.

[Reporting paper](<https://www.sota2.com/research/paper/69dc78a17d0172140ec78beb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69dc78a17d0172140ec78beb.md>)

Recorded evaluation configuration: {"Train Tokens (Embed Model)":"90088 / -","Train Tokens (Base Model)":"329236 / 2451","Train Tokens (Reflection Model)":"55382 / 57279","Inference Tokens (Base Model)":"252543 / 621"}. This record does not assert independent reproduction.

### DSPy GEPA

F1 Score: 59.29. Result ID: 69dcbdd547f34ce924c42036.

[Reporting paper](<https://www.sota2.com/research/paper/69dc78a17d0172140ec78beb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a3a1e209131cea81ee3.md>)

Recorded evaluation configuration: {"Train Tokens (Base Model)":"1321431 / 229379","Train Tokens (Reflection Model)":"24659 / 13374","Inference Tokens (Base Model)":"71669 / 1574"}. This record does not assert independent reproduction.

### TextGrad

F1 Score: 55.54. Result ID: 69dcbdd547f34ce924c42037.

[Reporting paper](<https://www.sota2.com/research/paper/69dc78a17d0172140ec78beb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73981e209131cea5d7cd.md>)

[Implementation](<https://github.com/zou-group/textgrad>)

Recorded evaluation configuration: {"Train Tokens (Base Model)":"1298401 / 14733","Train Tokens (Reflection Model)":"161048 / 177955","Inference Tokens (Base Model)":"123063 / 986"}. This record does not assert independent reproduction.

### DSPy BootstrapFewShot (#3)

F1 Score: 44.23. Result ID: 69dcbdd547f34ce924c42034.

[Reporting paper](<https://www.sota2.com/research/paper/69dc78a17d0172140ec78beb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70d01e209131cea4d60e.md>)

[Implementation](<https://github.com/stanfordnlp/dspy>)

Recorded evaluation configuration: {"Train Tokens (Base Model)":"16281 / 237","Inference Tokens (Base Model)":"37889 / 1475"}. This record does not assert independent reproduction.

### KNN

F1 Score: 35.08. Result ID: 69dcbdd547f34ce924c42032.

[Reporting paper](<https://www.sota2.com/research/paper/69dc78a17d0172140ec78beb.md>)

Recorded evaluation configuration: {"Inference Tokens (Base Model)":"34052 / 2965"}. This record does not assert independent reproduction.

### Initial prompt

F1 Score: 17.58. Result ID: 69dcbdd547f34ce924c42031.

[Reporting paper](<https://www.sota2.com/research/paper/69dc78a17d0172140ec78beb.md>)

Recorded evaluation configuration: {"Inference Tokens (Base Model)":"19263 / 2683"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.