Open-book generation under knowledge conflict on ConFiQA MR 1,500
59.8Ps ScoreContextDPO
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ContextDPOModel=LLAMA-3.1-8B, O&I Prompting=true2026.01 | 59.8 | 20.67 | 25.68 | |
| ContextDPOModel=LLAMA-3.1-8B, O&I Prompting=false2026.01 | 57.53 | 21.33 | 27.05 | |
| ContextDPOModel=MISTRAL-7B-INSTRUCT, O&I Prompting=true2026.01 | 57.33 | 21.47 | 27.24 | |
| ContextFocusModel=LLAMA-3.1-8B, O&I Prompting=false, Steering Layer=13, Steering Multiplier (m)=22026.01 | 54.47 | 16.07 | 22.78 | |
| ContextFocusModel=MISTRAL-7B-INSTRUCT, O&I Prompting=true, Steering Layer=11, Steering Multiplier (m)=22026.01 | 51.4 | 21.27 | 29.27 | |
| ContextFocusModel=LLAMA-3.1-8B, O&I Prompting=true, Steering Layer=13, Steering Multiplier (m)=22026.01 | 48.93 | 14.87 | 23.3 | |
| COIECDModel=MISTRAL-7B-INSTRUCT, O&I Prompting=true2026.01 | 45 | 22.33 | 33.17 | |
| ContextDPOModel=MISTRAL-7B-INSTRUCT, O&I Prompting=false2026.01 | 42.2 | 24.4 | 36.64 | |
| BaseModel=MISTRAL-7B-INSTRUCT, O&I Prompting=true2026.01 | 42 | 23.67 | 36.04 | |
| COIECDModel=LLAMA-3.1-8B, O&I Prompting=true2026.01 | 41.93 | 24.67 | 37.04 | |
| ContextFocusModel=MISTRAL-7B-INSTRUCT, O&I Prompting=false, Steering Layer=11, Steering Multiplier (m)=22026.01 | 40.47 | 24 | 37.22 | |
| BaseModel=LLAMA-3.1-8B, O&I Prompting=true2026.01 | 36.87 | 24.4 | 39.83 | |
| COIECDModel=LLAMA-3.1-8B, O&I Prompting=false2026.01 | 34.07 | 29 | 45.98 | |
| COIECDModel=MISTRAL-7B-INSTRUCT, O&I Prompting=false2026.01 | 32.87 | 29.6 | 47.39 | |
| BaseModel=LLAMA-3.1-8B, O&I Prompting=false2026.01 | 30.2 | 32.4 | 51.76 | |
| BaseModel=MISTRAL-7B-INSTRUCT, O&I Prompting=false2026.01 | 29.33 | 27.87 | 48.72 |