Retrieval Following on ConFiQA QA 1.0 (test)
92.3PcContext-DPO
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Context-DPOModel=LLAMA2-7B-CHAT2024.12 | 92.3 | 3.3 | 3.5 | 64.7 | |
| Context-DPOModel=MISTRAL-7B-INSTRUCT2024.12 | 78.6 | 11 | 12.3 | 10.7 | |
| O&IModel=LLAMA2-7B-CHAT2024.12 | 77.3 | 13.3 | 14.7 | 54.7 | |
| Context-DPOModel=QWEN2-7B-INSTRUCT2024.12 | 74.3 | 11.6 | 13.5 | 19.7 | |
| AttrModel=LLAMA2-7B-CHAT2024.12 | 72 | 14.7 | 16.9 | 1.3 | |
| Context-DPOModel=LLAMA3-8B-INSTRUCT2024.12 | 69.7 | 12.7 | 15.4 | 39.7 | |
| SFTModel=LLAMA2-7B-CHAT2024.12 | 63.2 | 25.2 | 28.5 | 0.4 | |
| BaseModel=LLAMA2-7B-CHAT2024.12 | 61.5 | 25.6 | 29.4 | 0.5 | |
| O&IModel=MISTRAL-7B-INSTRUCT2024.12 | 60.3 | 20 | 24.3 | 9.3 | |
| O&IModel=QWEN2-7B-INSTRUCT2024.12 | 58 | 20.3 | 26 | 5 | |
| AttrModel=MISTRAL-7B-INSTRUCT2024.12 | 44.4 | 30.3 | 40.6 | 2 | |
| SFTModel=MISTRAL-7B-INSTRUCT2024.12 | 39.4 | 40.1 | 50.4 | 0.5 | |
| BaseModel=MISTRAL-7B-INSTRUCT2024.12 | 39.3 | 40.5 | 50.8 | 0.3 | |
| AttrModel=QWEN2-7B-INSTRUCT2024.12 | 38.3 | 35 | 47.7 | 0.3 | |
| SFTModel=LLAMA3-8B-INSTRUCT2024.12 | 36.9 | 42.2 | 53.3 | 0 | |
| BaseModel=LLAMA3-8B-INSTRUCT2024.12 | 35.8 | 44.3 | 55.3 | 0 | |
| O&IModel=LLAMA3-8B-INSTRUCT2024.12 | 32.7 | 30.3 | 48.2 | 1.3 | |
| AttrModel=LLAMA3-8B-INSTRUCT2024.12 | 25.7 | 41 | 61.5 | 0 | |
| SFTModel=QWEN2-7B-INSTRUCT2024.12 | 24.8 | 42.7 | 63.3 | 0 | |
| BaseModel=QWEN2-7B-INSTRUCT2024.12 | 24 | 43.3 | 65.4 | 0 |