Question Answering on SearchQA (Accuracy, Consistency Rate, Overconfidence Rate)
95.1AccuracyRAG
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RAGModel=Llama-3.1-8B, Privacy Budget=εtotal = ∞2025.10 | 95.1 | — | — | |
| RAGModel=Gemma-2-2B, Privacy Budget=εtotal = ∞2025.10 | 94.12 | — | — | |
| RAGModel=Phi-4-mini, Privacy Budget=εtotal = ∞2025.10 | 92.16 | — | — | |
| DP-SynRAGModel=Llama-3.1-8B, Privacy Budget=εtotal = 102025.10 | 91.18 | — | — | |
| DP-SynRAGModel=Phi-4-mini, Privacy Budget=εtotal = 102025.10 | 89.61 | — | — | |
| Non-RAGModel=Llama-3.1-8B, Privacy Budget=εtotal = 02025.10 | 88.24 | — | — | |
| DP-SynRAGModel=Gemma-2-2B, Privacy Budget=εtotal = 102025.10 | 85.1 | — | — | |
| DP-RAGModel=Phi-4-mini, Privacy Budget=εquery = 10 (εtotal ≈ 1000)2025.10 | 85.1 | — | — | |
| DP-RAGModel=Llama-3.1-8B, Privacy Budget=εquery = 10 (εtotal ≈ 1000)2025.10 | 84.9 | — | — | |
| DP-RAGModel=Gemma-2-2B, Privacy Budget=εquery = 10 (εtotal ≈ 1000)2025.10 | 83.14 | — | — | |
| LoRABase Model=Qwen3.5–4B2026.06 | 78.6 | — | — | |
| SOFTSKILLBase Model=Qwen3.5–4B, Insertion Placement=prompt start2026.06 | 76.4 | — | — | |
| SOFTSKILLBase Model=Qwen3.5–4B, Insertion Placement=skill section2026.06 | 76.4 | — | — | |
| SOFTSKILL-meanBase Model=Qwen3.5–4B, Insertion Placement=prompt start2026.06 | 72.6 | — | — | |
| SOFTSKILL-meanBase Model=Qwen3.5–4B, Insertion Placement=skill section2026.06 | 72.6 | — | — | |
| DGAOModel=Qwen-2.5-3B2026.05 | 71.87 | 87.81 | 19.26 | |
| DGAOModel=Qwen-2.5-14B2026.05 | 71.22 | 89.82 | 19.42 | |
| SkillOptBase Model=Qwen3.5–4B2026.06 | 71.2 | — | — | |
| Non-RAGModel=Gemma-2-2B, Privacy Budget=εtotal = 02025.10 | 70.59 | — | — | |
| SOFTSKILL-mean-init-onlyBase Model=Qwen3.5–4B, Insertion Placement=prompt start2026.06 | 70.5 | — | — | |
| PAFTModel=Qwen-2.5-3B2026.05 | 69.62 | 90.91 | 24.1 | |
| PAFTModel=Qwen-2.5-14B2026.05 | 69.03 | 92.71 | 24.08 | |
| GEPABase Model=Qwen3.5–4B2026.06 | 68.6 | — | — | |
| Trace2SkillBase Model=Qwen3.5–4B2026.06 | 68.5 | — | — | |
| SFTModel=Qwen-2.5-3B2026.05 | 68.49 | 85.54 | 20.64 | |
| No skillBase Model=Qwen3.5–4B2026.06 | 68.1 | — | — | |
| SFTModel=Qwen-2.5-14B2026.05 | 67.9 | 87.46 | 20.65 | |
| Human skillBase Model=Qwen3.5–4B2026.06 | 66.3 | — | — | |
| Non-RAGModel=Phi-4-mini, Privacy Budget=εtotal = 02025.10 | 65.69 | — | — | |
| LLM skillBase Model=Qwen3.5–4B2026.06 | 65 | — | — | |
| DGAOModel=Gemma-3-4B2026.05 | 63.1 | 80.6 | 22.11 | |
| DGAOModel=Llama-3.2-3B2026.05 | 62.51 | 84.72 | 25.43 | |
| DGAOModel=Llama-3.2-11B2026.05 | 62.13 | 86.54 | 25.52 | |
| TextGradBase Model=Qwen3.5–4B2026.06 | 60.7 | — | — | |
| DP-SynthModel=Phi-4-mini, Privacy Budget=εtotal = 102025.10 | 60.2 | — | — | |
| SFTModel=Gemma-3-4B2026.05 | 59.72 | 78.09 | 23 | |
| SFTModel=Llama-3.2-3B2026.05 | 58.72 | 81.21 | 25.88 | |
| PAFTModel=Gemma-3-4B2026.05 | 58.64 | 83.82 | 29.74 | |
| SFTModel=Llama-3.2-11B2026.05 | 58.44 | 83.16 | 25.96 | |
| PAFTModel=Llama-3.2-3B2026.05 | 56.34 | 86.45 | 33.49 | |
| PAFTModel=Llama-3.2-11B2026.05 | 56.18 | 85.27 | 31.28 | |
| DP-SynthModel=Llama-3.1-8B, Privacy Budget=εtotal = 102025.10 | 40 | — | — | |
| DP-SynthModel=Gemma-2-2B, Privacy Budget=εtotal = 102025.10 | 20.2 | — | — |