Access Control on Synthetic D&D dataset (test)
92AccuracyStatic Filter
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Static FilterBackbone=Llama3.1-8B, Approach=Rule-based/RBAC2025.12 | 92 | 94 | |
| Static FilterBackbone=Qwen2.5-14B, Approach=Rule-based/RBAC2025.12 | 91 | 93 | |
| Static FilterBackbone=Phi4-14B, Approach=Rule-based/RBAC2025.12 | 90 | 93 | |
| Static FilterBackbone=DeepSeek-r1, Approach=Rule-based/RBAC2025.12 | 90 | 93 | |
| ARBITERBackbone=Qwen2.5-14B, Approach=Few-shot filtering2025.12 | 85 | 89 | |
| ARBITERBackbone=Phi4-14B, Approach=Few-shot filtering2025.12 | 82 | 87 | |
| ARBITERBackbone=DeepSeek-r1, Approach=Few-shot filtering2025.12 | 78 | 84 | |
| ARBITERBackbone=Llama3.1-8B, Approach=Few-shot filtering2025.12 | 75 | 83 |