Privacy-preserving dialogue processing on PUPA-TNB (test)
88.2QualityGPT-4o-mini
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-4o-miniVariant=Unredacted2025.10 | 88.2 | 100 | — | — | |
| Privacy-R1Backbone=Llama-3.1-8B-Instruct2025.10 | 86.1 | 4.5 | 0.6 | 3 | |
| PAPILLONBackbone=Llama-3.1-8B-Instruct2025.10 | 85.5 | 7.5 | — | — | |
| Privacy-R1Backbone=Qwen2-7B-Instruct2025.10 | 81.1 | 10.5 | 3.1 | 4.7 | |
| Privacy-R1Backbone=Mistral-7B-Instruct-v0.32025.10 | 80.2 | 8.1 | 2.6 | 3.8 | |
| Heuristic Router (SFT-only)Backbone=Llama-3.1-8B-Instruct2025.10 | 78.5 | 1 | — | — | |
| PAPILLONBackbone=Qwen2-7B-Instruct2025.10 | 78 | 15.2 | — | — | |
| PAPILLONBackbone=Mistral-7B-Instruct-v0.32025.10 | 77.6 | 11.9 | — | — | |
| GPT-4o-miniVariant=Redacted2025.10 | 77.2 | 0 | — | — | |
| Heuristic Router (SFT-only)Backbone=Qwen2-7B-Instruct2025.10 | 77.2 | 1.5 | — | — | |
| Heuristic Router (SFT-only)Backbone=Mistral-7B-Instruct-v0.32025.10 | 76.8 | 1.2 | — | — | |
| Always-LocalBackbone=Qwen2-7B-Instruct2025.10 | 76.5 | 0 | — | — | |
| Always-LocalBackbone=Mistral-7B-Instruct-v0.32025.10 | 75.7 | 0 | — | — | |
| Always-LocalBackbone=Llama-3.1-8B-Instruct2025.10 | 71.8 | 0 | — | — | |
| Privacy-R1Backbone=Llama-3.2-3B-Instruct2025.10 | 65.2 | 19.5 | 4.3 | 5.4 | |
| Privacy-R1Backbone=Llama-3.2-1B-Instruct2025.10 | 62.5 | 25 | 4.5 | 14.3 | |
| PAPILLONBackbone=Llama-3.2-3B-Instruct2025.10 | 60.9 | 24.9 | — | — | |
| Heuristic Router (SFT-only)Backbone=Llama-3.2-3B-Instruct2025.10 | 59.1 | 1.8 | — | — | |
| PAPILLONBackbone=Llama-3.2-1B-Instruct2025.10 | 58 | 39.3 | — | — | |
| Always-LocalBackbone=Llama-3.2-3B-Instruct2025.10 | 57.3 | 0 | — | — | |
| Heuristic Router (SFT-only)Backbone=Llama-3.2-1B-Instruct2025.10 | 52.5 | 2 | — | — | |
| Always-LocalBackbone=Llama-3.2-1B-Instruct2025.10 | 41.2 | 0 | — | — |