Query-Unrelated PII Masking on PII-Real
83.8F1 ScoreNaive w/ Choice
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Naive w/ ChoiceModel=GPT4o, Category=w/ Extra Information2025.02 | 83.8 | 84.1 | |
| Naive w/ ChoiceModel=Qwen2.5, Category=w/ Extra Information2025.02 | 81.2 | 81.5 | |
| Naive w/ ChoiceModel=Llama3.1, Category=w/ Extra Information2025.02 | 79.6 | 79.9 | |
| Self-ConsistencyModel=GPT4o, Category=Advanced Method2025.02 | 79.4 | 79.6 | |
| Self-CoTModel=GPT4o, Category=Advanced Method2025.02 | 78.2 | 78.5 | |
| Auto-CoTModel=Qwen2.5, Category=Advanced Method2025.02 | 78.2 | 78.5 | |
| Auto-CoTModel=Llama3.1, Category=Advanced Method2025.02 | 78.1 | 78.4 | |
| Auto-CoTModel=GPT4o, Category=Advanced Method2025.02 | 77.1 | 77.3 | |
| Self-CoTModel=Llama3.1, Category=Advanced Method2025.02 | 76.8 | 77.1 | |
| PS-CoTModel=GPT4o, Category=Advanced Method2025.02 | 75.8 | 76.1 | |
| PS-CoTModel=Qwen2.5, Category=Advanced Method2025.02 | 75.3 | 75.6 | |
| Self-CoTModel=Qwen2.5, Category=Advanced Method2025.02 | 75.1 | 75.4 | |
| PS-CoTModel=Llama3.1, Category=Advanced Method2025.02 | 74.5 | 74.8 | |
| NaiveModel=GPT4o, Category=Basic Method2025.02 | 74.3 | 74.5 | |
| NaiveModel=Llama3.1, Category=Basic Method2025.02 | 73.8 | 74.2 | |
| Self-ConsistencyModel=Qwen2.5, Category=Advanced Method2025.02 | 73.4 | 73.8 | |
| Self-ConsistencyModel=Llama3.1, Category=Advanced Method2025.02 | 73.2 | 73.6 | |
| NaiveModel=Qwen2.5, Category=Basic Method2025.02 | 72.1 | 72.5 | |
| Naive w/ ChoiceModel=Qwen2.5-SLM, Category=w/ Extra Information2025.02 | 68.9 | 72.5 | |
| Auto-CoTModel=Llama3.1-SLM, Category=Advanced Method2025.02 | 58.9 | 59.3 | |
| PS-CoTModel=Qwen2.5-SLM, Category=Advanced Method2025.02 | 57.8 | 61.4 | |
| Auto-CoTModel=Qwen2.5-SLM, Category=Advanced Method2025.02 | 56.2 | 59.7 | |
| Self-CoTModel=Qwen2.5-SLM, Category=Advanced Method2025.02 | 56.1 | 59.6 | |
| NaiveModel=Qwen2.5-SLM, Category=Basic Method2025.02 | 55.8 | 59.3 | |
| Self-CoTModel=Llama3.1-SLM, Category=Advanced Method2025.02 | 54.7 | 55.1 | |
| Self-ConsistencyModel=Qwen2.5-SLM, Category=Advanced Method2025.02 | 51.2 | 54.8 | |
| Self-ConsistencyModel=Llama3.1-SLM, Category=Advanced Method2025.02 | 50.8 | 51.3 | |
| PS-CoTModel=Llama3.1-SLM, Category=Advanced Method2025.02 | 49.8 | 50.3 | |
| Naive w/ ChoiceModel=Llama3.1-SLM, Category=w/ Extra Information2025.02 | 47.8 | 48.3 | |
| NaiveModel=Llama3.1-SLM, Category=Basic Method2025.02 | 43.6 | 44.1 |