Retrieval Following on ConFiQA MC 1.0 (test)
54.9PcContext-DPO
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Context-DPOModel=QWEN2-7B-INSTRUCT2024.12 | 54.9 | 21.3 | 27.9 | 2,190 | |
| Context-DPOModel=LLAMA2-7B-CHAT2024.12 | 52.7 | 12.3 | 19 | 3,200 | |
| O&IModel=LLAMA2-7B-CHAT2024.12 | 50.7 | 16 | 24 | 2,930 | |
| Context-DPOModel=LLAMA3-8B-INSTRUCT2024.12 | 48.9 | 18.6 | 28.1 | 1,500 | |
| Context-DPOModel=MISTRAL-7B-INSTRUCT2024.12 | 46.7 | 18.3 | 28.1 | 960 | |
| SFTModel=LLAMA2-7B-CHAT2024.12 | 39.3 | 24.5 | 38.4 | 10 | |
| BaseModel=LLAMA2-7B-CHAT2024.12 | 38.8 | 24.7 | 38.5 | 10 | |
| AttrModel=LLAMA2-7B-CHAT2024.12 | 33.7 | 25 | 43.6 | 10 | |
| O&IModel=QWEN2-7B-INSTRUCT2024.12 | 31 | 26 | 45.6 | 530 | |
| O&IModel=MISTRAL-7B-INSTRUCT2024.12 | 26.7 | 22.7 | 46 | 330 | |
| SFTModel=LLAMA3-8B-INSTRUCT2024.12 | 23.5 | 30.1 | 56.2 | 0 | |
| BaseModel=LLAMA3-8B-INSTRUCT2024.12 | 21.7 | 33.4 | 60.7 | 0 | |
| BaseModel=MISTRAL-7B-INSTRUCT2024.12 | 14.1 | 29.8 | 67.9 | 0 | |
| SFTModel=MISTRAL-7B-INSTRUCT2024.12 | 13.8 | 27.5 | 66.5 | 0 | |
| AttrModel=QWEN2-7B-INSTRUCT2024.12 | 13.7 | 30.7 | 69.2 | 0 | |
| O&IModel=LLAMA3-8B-INSTRUCT2024.12 | 13 | 25 | 65.8 | 30 | |
| AttrModel=MISTRAL-7B-INSTRUCT2024.12 | 13 | 28.7 | 68.8 | 70 | |
| SFTModel=QWEN2-7B-INSTRUCT2024.12 | 11.5 | 42.5 | 78.7 | 0 | |
| BaseModel=QWEN2-7B-INSTRUCT2024.12 | 10 | 43.7 | 81.4 | 0 | |
| AttrModel=LLAMA3-8B-INSTRUCT2024.12 | 9.7 | 31.3 | 76.4 | 0 |