Personally Identifiable Information (PII) Entity Recognition on SPY Legal Questions
93.2NameDeBERTa-v3†
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| DeBERTa-v3†Params=184 M, Task=token cls, Open-labels=no, Fine-tuned=yes2026.05 | 93.2 | 99.1 | 98 | 99 | 96.6 | 98.7 | 94.5 | |
| GG OmniParams=209 M, Task=span + text cls, Open-labels=yes, Fine-tuned=yes2026.05 | 88.8 | 95.9 | 43.8 | 19.5 | 80.9 | 89.4 | 95.5 | |
| PresidioParams=-, Task=token cls + rules, Open-labels=rules only, Fine-tuned=no2026.05 | 79.4 | 91.8 | — | 21.3 | 34.4 | 68.1 | — | |
| GG (uni)Params=147 M, Task=span + text cls, Open-labels=yes, Fine-tuned=no2026.05 | 73.6 | 96 | 40.9 | — | 80.7 | 86.8 | 83.4 | |
| GG (bi)Params=145 M, Task=span + text cls, Open-labels=yes, Fine-tuned=no2026.05 | 72.6 | 96 | 43.9 | — | 75.8 | 90.4 | 83.8 | |
| Llama-3-70BParams=70 B, Task=autoregressive, Open-labels=yes, Fine-tuned=no2026.05 | 68.9 | 88.5 | 59.7 | 92.5 | 62.2 | 92.8 | 81.3 |