Privacy-preserving Query Rewriting on DelegateCI-Bench Medical
79.5UtilizationDirect LLM
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Direct LLMRewriter=raw query2026.06 | 79.5 | — | — | — | 89.8 | — | |
| CI-guided reinforcement learning frameworkRewriter=Qwen-2.5-3B (shared), Local aggregator=Llama-3.1-8B-Instruct2026.06 | 66.7 | 4.2 | 1.5 | 71 | 80.5 | 6.6 | |
| Presidio(PII Only)Rewriter=rule-based PII2026.06 | 64.2 | 1.8 | 0.6 | 95.3 | 76.7 | — | |
| PUFTRewriter=Llama-3.1-8B, Local aggregator=Llama-3.1-8B-Instruct2026.06 | 62.8 | 28.5 | 9.5 | 41.9 | 77.8 | 3.8 | |
| CI-guided reinforcement learning frameworkRewriter=Qwen-2.5-3B (shared), Local aggregator=Qwen-2.5-7B-Instruct2026.06 | 62 | 4.2 | 1.5 | 71 | 78.7 | 10.1 | |
| Local OnlyRewriter=original query, Local aggregator=Llama-3.1-8B-Instruct2026.06 | 60.7 | — | — | — | 74 | 0 | |
| PapillonRewriter=Qwen-2.5-7B, Local aggregator=Qwen-2.5-7B-Instruct2026.06 | 59 | 55 | 23.9 | 80.6 | 76.3 | 7.7 | |
| PapillonRewriter=Llama-3.1-8B, Local aggregator=Llama-3.1-8B-Instruct2026.06 | 49.2 | 46.7 | 16.7 | 61.7 | 70.7 | 3.3 | |
| PUFTRewriter=Qwen-2.5-7B, Local aggregator=Qwen-2.5-7B-Instruct2026.06 | 49 | 20.3 | 7.2 | 27.8 | 72.5 | 3.9 | |
| Local OnlyRewriter=original query, Local aggregator=Qwen-2.5-7B-Instruct2026.06 | 47.3 | — | — | — | 68.6 | 0 | |
| CI-guided reinforcement learning frameworkRewriter=Qwen-2.5-3B (shared), Local aggregator=Qwen-2.5-1.5B-Instruct2026.06 | 41.2 | 4.2 | 1.5 | 71 | 65.4 | 12.5 | |
| PapillonRewriter=Qwen-2.5-1.5B, Local aggregator=Qwen-2.5-1.5B-Instruct2026.06 | 35 | 62.7 | 40.8 | 61.5 | 57.7 | 4.8 | |
| Local OnlyRewriter=original query, Local aggregator=Qwen-2.5-1.5B-Instruct2026.06 | 34 | — | — | — | 52.9 | 0 | |
| PUFTRewriter=Qwen-2.5-1.5B, Local aggregator=Qwen-2.5-1.5B-Instruct2026.06 | 32.5 | 16.7 | 9.9 | 13.1 | 56.3 | 3.4 |