Knowledge Conflict Resolution on PAVE (test)
59IEJSD-based arbitration
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| JSD-based arbitrationBackbone=Llama3-8B, K=5, Temperature=1.02026.05 | 59 | 65.2 | 62.1 | |
| ContextEvalBackbone=Qwen3-32B2026.05 | 58.49 | 64.25 | 61.47 | |
| ExplicitSCRBackbone=Gemini-2.5-flash2026.05 | 57.5 | 66.93 | 62.22 | |
| JSD-based arbitrationBackbone=Qwen3-32B, K=5, Temperature=1.02026.05 | 56.8 | 69.12 | 62.97 | |
| ContextEvalBackbone=Gemini-2.5-flash2026.05 | 55.61 | 38.73 | 47.17 | |
| JSD-based arbitrationBackbone=Gpt-4o-mini, K=5, Temperature=1.02026.05 | 55.38 | 67.14 | 61.26 | |
| ContextEvalBackbone=Gpt-4o-mini2026.05 | 55.2 | 67.03 | 61.12 | |
| ContextEvalBackbone=Llama3-8B2026.05 | 54.92 | 61.17 | 58.05 | |
| ExplicitSCRBackbone=Gpt-4o-mini2026.05 | 54.12 | 64.35 | 59.24 | |
| InternalEvalBackbone=Qwen3-32B2026.05 | 52.63 | 59.29 | 55.96 | |
| ContextEvalBackbone=Phi-42026.05 | 52.5 | 66.6 | 59.55 | |
| JSD-based arbitrationBackbone=Phi-4, K=5, Temperature=1.02026.05 | 52.4 | 63.6 | 58 | |
| ContextConfBackbone=Phi-42026.05 | 52.2 | 52.2 | 52.2 | |
| InternalEvalBackbone=Gpt-4o-mini2026.05 | 51.43 | 57.6 | 54.52 | |
| JSD-based arbitrationBackbone=Mistral-7B, K=5, Temperature=1.02026.05 | 51.2 | 62.96 | 57.08 | |
| TPCBackbone=Phi-42026.05 | 51.14 | 51.14 | 51.14 | |
| JSD-based arbitrationBackbone=Gemini-2.5-flash, K=5, Temperature=1.02026.05 | 51.12 | 65.01 | 58.07 | |
| InternalEvalBackbone=Llama3-8B2026.05 | 51.04 | 58.39 | 54.72 | |
| ImplicitSCRBackbone=Llama3-8B2026.05 | 50.4 | 60 | 55.2 | |
| ImplicitSCRBackbone=Phi-42026.05 | 50.1 | 60.8 | 55.45 | |
| InternalEvalBackbone=Gemini-2.5-flash2026.05 | 49.95 | 55.81 | 52.88 | |
| ContextConfBackbone=Gpt-4o-mini2026.05 | 47.9 | 64.7 | 56.3 | |
| ImplicitSCRBackbone=Qwen3-32B2026.05 | 47.77 | 60.08 | 53.93 | |
| InternalConfBackbone=Gpt-4o-mini2026.05 | 47.6 | 48.4 | 48 | |
| InternalEvalBackbone=Phi-42026.05 | 47.6 | 48.8 | 48.2 | |
| TPCBackbone=Gpt-4o-mini2026.05 | 46.9 | 61.6 | 54.25 | |
| InternalConfBackbone=Phi-42026.05 | 46.6 | 65.2 | 55.9 | |
| ImplicitSCRBackbone=Gpt-4o-mini2026.05 | 45.48 | 59.88 | 52.68 | |
| TACS-LRBackbone=Phi-42026.05 | 45.2 | 51.8 | 47.15 | |
| TACS-LRBackbone=Qwen3-32B2026.05 | 45.2 | 51.8 | 48.5 | |
| TACS-LRBackbone=Gpt-4o-mini2026.05 | 44.6 | 54 | 49.3 | |
| TPCBackbone=Mistral-7B2026.05 | 43.33 | 57.67 | 50.5 | |
| ExplicitSCRBackbone=Phi-42026.05 | 41.81 | 50.84 | 46.33 | |
| TACS-LRBackbone=Gemini-2.5-flash2026.05 | 39.5 | 53 | 46.25 | |
| TACS-LRBackbone=Mistral-7B2026.05 | 37.2 | 47.8 | 42.5 | |
| ImplicitSCRBackbone=Mistral-7B2026.05 | 34.56 | 38.83 | 36.7 | |
| InternalConfBackbone=Mistral-7B2026.05 | 34.33 | 34.33 | 34.33 | |
| ExplicitSCRBackbone=Llama3-8B2026.05 | 32.6 | 38.6 | 35.6 | |
| TACS-LRBackbone=Llama3-8B2026.05 | 28.6 | 34.6 | 31.6 | |
| ContextEvalBackbone=Mistral-7B2026.05 | 27.61 | 10.33 | 18.97 | |
| ExplicitSCRBackbone=Qwen3-32B2026.05 | 19.27 | 23.55 | 21.41 | |
| ContextConfBackbone=Mistral-7B2026.05 | 15.33 | 10.33 | 12.83 | |
| ImplicitSCRBackbone=Gemini-2.5-flash2026.05 | 14.9 | 21.85 | 18.38 | |
| ExplicitSCRBackbone=Mistral-7B2026.05 | 10.33 | 18.77 | 14.55 | |
| InternalEvalBackbone=Mistral-7B2026.05 | 6.26 | 3.08 | 4.67 |