Query-Related PII Detection on PII-Real (test)
86.1F1 ScoreNaive w/ Choice
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Naive w/ ChoiceModel=GPT4o, Prompting Strategy=Naive w/ Choice, Category=w/ Extra Information2025.02 | 86.1 | 86.1 | |
| Naive w/ ChoiceModel=Qwen2.5, Prompting Strategy=Naive w/ Choice, Category=w/ Extra Information2025.02 | 84.7 | 84.9 | |
| Naive w/ ChoiceModel=Qwen2.5-SLM, Prompting Strategy=Naive w/ Choice, Category=w/ Extra Information2025.02 | 79.1 | 79.3 | |
| Naive w/ ChoiceModel=Llama3.1, Prompting Strategy=Naive w/ Choice, Category=w/ Extra Information2025.02 | 78.2 | 78.4 | |
| Self-ConsistencyModel=GPT4o, Prompting Strategy=Self-Consistency, Category=Advanced Method2025.02 | 74.5 | 74.7 | |
| Self-CoTModel=GPT4o, Prompting Strategy=Self-CoT, Category=Advanced Method2025.02 | 73.8 | 74.1 | |
| Auto-CoTModel=Qwen2.5, Prompting Strategy=Auto-CoT, Category=Advanced Method2025.02 | 72.9 | 73.2 | |
| Auto-CoTModel=Llama3.1, Prompting Strategy=Auto-CoT, Category=Advanced Method2025.02 | 72.4 | 72.8 | |
| Self-CoTModel=Llama3.1, Prompting Strategy=Self-CoT, Category=Advanced Method2025.02 | 71.2 | 71.4 | |
| Self-CoTModel=Qwen2.5, Prompting Strategy=Self-CoT, Category=Advanced Method2025.02 | 69.4 | 69.7 | |
| Auto-CoTModel=GPT4o, Prompting Strategy=Auto-CoT, Category=Advanced Method2025.02 | 69.1 | 69.3 | |
| PS-CoTModel=Qwen2.5, Prompting Strategy=PS-CoT, Category=Advanced Method2025.02 | 68.9 | 69.1 | |
| PS-CoTModel=GPT4o, Prompting Strategy=PS-CoT, Category=Advanced Method2025.02 | 68.3 | 68.5 | |
| PS-CoTModel=Llama3.1, Prompting Strategy=PS-CoT, Category=Advanced Method2025.02 | 67.3 | 67.6 | |
| Self-ConsistencyModel=Qwen2.5, Prompting Strategy=Self-Consistency, Category=Advanced Method2025.02 | 67.1 | 67.4 | |
| Self-ConsistencyModel=Llama3.1, Prompting Strategy=Self-Consistency, Category=Advanced Method2025.02 | 65.6 | 65.9 | |
| NaiveModel=GPT4o, Prompting Strategy=Naive, Category=Basic Method2025.02 | 65.2 | 65.4 | |
| NaiveModel=Llama3.1, Prompting Strategy=Naive, Category=Basic Method2025.02 | 64.8 | 65.1 | |
| NaiveModel=Qwen2.5, Prompting Strategy=Naive, Category=Basic Method2025.02 | 63.5 | 63.8 | |
| Naive w/ ChoiceModel=Llama3.1-SLM, Prompting Strategy=Naive w/ Choice, Category=w/ Extra Information2025.02 | 53.8 | 54.1 | |
| PS-CoTModel=Qwen2.5-SLM, Prompting Strategy=PS-CoT, Category=Advanced Method2025.02 | 46.8 | 47.1 | |
| Auto-CoTModel=Llama3.1-SLM, Prompting Strategy=Auto-CoT, Category=Advanced Method2025.02 | 45.1 | 45.4 | |
| NaiveModel=Qwen2.5-SLM, Prompting Strategy=Naive, Category=Basic Method2025.02 | 42.7 | 42.9 | |
| Self-CoTModel=Qwen2.5-SLM, Prompting Strategy=Self-CoT, Category=Advanced Method2025.02 | 41.8 | 42.1 | |
| Self-CoTModel=Llama3.1-SLM, Prompting Strategy=Self-CoT, Category=Advanced Method2025.02 | 40.8 | 41.1 | |
| Auto-CoTModel=Qwen2.5-SLM, Prompting Strategy=Auto-CoT, Category=Advanced Method2025.02 | 38.9 | 39.2 | |
| PS-CoTModel=Llama3.1-SLM, Prompting Strategy=PS-CoT, Category=Advanced Method2025.02 | 36.8 | 37.2 | |
| NaiveModel=Llama3.1-SLM, Prompting Strategy=Naive, Category=Basic Method2025.02 | 34.6 | 34.8 | |
| Self-ConsistencyModel=Qwen2.5-SLM, Prompting Strategy=Self-Consistency, Category=Advanced Method2025.02 | 33.8 | 34.2 | |
| Self-ConsistencyModel=Llama3.1-SLM, Prompting Strategy=Self-Consistency, Category=Advanced Method2025.02 | 32.7 | 33.1 |