Open-book generation under knowledge conflict on ConFiQA 1,500 subset
81.07Ps ScoreContextDPO
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ContextDPOModel=LLAMA-3.1-8B, O&I Prompting=true2026.01 | 81.07 | 8.87 | 9.86 | |
| ContextFocusModel=LLAMA-3.1-8B, O&I Prompting=true, Steering Layer=13, Steering Multiplier (m)=22026.01 | 77.53 | 7.6 | 8.93 | |
| ContextDPOModel=MISTRAL-7B-INSTRUCT, O&I Prompting=true2026.01 | 77.4 | 10.13 | 11.58 | |
| ContextFocusModel=MISTRAL-7B-INSTRUCT, O&I Prompting=true, Steering Layer=11, Steering Multiplier (m)=22026.01 | 75.13 | 11.53 | 13.31 | |
| ContextDPOModel=MISTRAL-7B-INSTRUCT, O&I Prompting=false2026.01 | 72.93 | 13.33 | 15.46 | |
| COIECDModel=MISTRAL-7B-INSTRUCT, O&I Prompting=true2026.01 | 71 | 13.93 | 16.41 | |
| ContextFocusModel=LLAMA-3.1-8B, O&I Prompting=false, Steering Layer=13, Steering Multiplier (m)=22026.01 | 70.87 | 9.27 | 11.56 | |
| BaseModel=LLAMA-3.1-8B, O&I Prompting=true2026.01 | 70.4 | 15.4 | 17.95 | |
| COIECDModel=LLAMA-3.1-8B, O&I Prompting=true2026.01 | 69.67 | 14.27 | 17 | |
| ContextFocusModel=MISTRAL-7B-INSTRUCT, O&I Prompting=false, Steering Layer=11, Steering Multiplier (m)=22026.01 | 67.8 | 16.73 | 19.79 | |
| BaseModel=MISTRAL-7B-INSTRUCT, O&I Prompting=true2026.01 | 66.27 | 15.53 | 18.99 | |
| COIECDModel=MISTRAL-7B-INSTRUCT, O&I Prompting=false2026.01 | 62.13 | 21.87 | 26.03 | |
| ContextDPOModel=LLAMA-3.1-8B, O&I Prompting=true2026.01 | 59.13 | 17.33 | 22.67 | |
| BaseModel=MISTRAL-7B-INSTRUCT, O&I Prompting=false2026.01 | 58.93 | 22.87 | 27.95 | |
| ContextDPOModel=LLAMA-3.1-8B, O&I Prompting=false2026.01 | 58.4 | 18.6 | 24.16 | |
| ContextDPOModel=MISTRAL-7B-INSTRUCT, O&I Prompting=true2026.01 | 56.93 | 17.2 | 23.2 | |
| ContextFocusModel=LLAMA-3.1-8B, O&I Prompting=false, Steering Layer=13, Steering Multiplier (m)=22026.01 | 53.33 | 10.2 | 16.05 | |
| ContextFocusModel=MISTRAL-7B-INSTRUCT, O&I Prompting=true, Steering Layer=11, Steering Multiplier (m)=22026.01 | 50.6 | 15.53 | 23.49 | |
| ContextFocusModel=LLAMA-3.1-8B, O&I Prompting=true, Steering Layer=13, Steering Multiplier (m)=22026.01 | 48.6 | 10.27 | 17.44 | |
| ContextDPOModel=LLAMA-3.1-8B, O&I Prompting=false2026.01 | 46.27 | 17.2 | 27.1 | |
| COIECDModel=MISTRAL-7B-INSTRUCT, O&I Prompting=true2026.01 | 42.47 | 16.4 | 27.86 | |
| BaseModel=MISTRAL-7B-INSTRUCT, O&I Prompting=true2026.01 | 37.07 | 16.67 | 31.02 | |
| COIECDModel=LLAMA-3.1-8B, O&I Prompting=true2026.01 | 36.6 | 17.53 | 32.39 | |
| COIECDModel=LLAMA-3.1-8B, O&I Prompting=false2026.01 | 35.47 | 32.6 | 47.89 | |
| BaseModel=LLAMA-3.1-8B, O&I Prompting=false2026.01 | 35.27 | 32.33 | 47.83 | |
| ContextFocusModel=MISTRAL-7B-INSTRUCT, O&I Prompting=false, Steering Layer=11, Steering Multiplier (m)=22026.01 | 35.13 | 19.33 | 35.5 | |
| ContextDPOModel=MISTRAL-7B-INSTRUCT, O&I Prompting=false2026.01 | 32.87 | 18.47 | 35.97 | |
| BaseModel=LLAMA-3.1-8B, O&I Prompting=true2026.01 | 30 | 18.53 | 38.19 | |
| COIECDModel=MISTRAL-7B-INSTRUCT, O&I Prompting=false2026.01 | 25.93 | 18.4 | 41.5 | |
| BaseModel=MISTRAL-7B-INSTRUCT, O&I Prompting=false2026.01 | 19.67 | 19.33 | 49.57 | |
| COIECDModel=LLAMA-3.1-8B, O&I Prompting=false2026.01 | 14.8 | 19.8 | 57.23 | |
| BaseModel=LLAMA-3.1-8B, O&I Prompting=false2026.01 | 12.47 | 17.2 | 57.98 |