Retrieval Following on Natural Questions (test)
98.4PcContext-DPO
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Context-DPOBase Model=LLAMA3-8B-INSTRUCT2024.12 | 98.4 | 8.5 | 7.9 | 3.1 | |
| Context-DPOBase Model=MISTRAL-7B-INSTRUCT2024.12 | 94.7 | 11.9 | 11.2 | 2.7 | |
| Context-DPOBase Model=QWEN2-7B-INSTRUCT2024.12 | 92.5 | 13.2 | 12.4 | 1.8 | |
| Context-DPOBase Model=LLAMA2-7B-CHAT2024.12 | 88.9 | 1.4 | 1.3 | 53.3 | |
| O&IBase Model=LLAMA3-8B-INSTRUCT2024.12 | 87.4 | 26 | 22.9 | 1.4 | |
| O&IBase Model=QWEN2-7B-INSTRUCT2024.12 | 87 | 25.5 | 22.7 | 1.6 | |
| AttrBase Model=LLAMA3-8B-INSTRUCT2024.12 | 86.3 | 55.9 | 39.3 | 0.1 | |
| O&IBase Model=MISTRAL-7B-INSTRUCT2024.12 | 85 | 19.9 | 18.8 | 1.8 | |
| O&IBase Model=LLAMA2-7B-CHAT2024.12 | 77.8 | 13.9 | 15.1 | 13.7 | |
| AttrBase Model=QWEN2-7B-INSTRUCT2024.12 | 75.8 | 36.7 | 32.6 | 0 | |
| AttrBase Model=MISTRAL-7B-INSTRUCT2024.12 | 74.3 | 35.6 | 32.4 | 0.2 | |
| AttrBase Model=LLAMA2-7B-CHAT2024.12 | 66.2 | 23.8 | 26.4 | 4.7 | |
| SFTBase Model=LLAMA3-8B-INSTRUCT2024.12 | 61.8 | 66.5 | 51.8 | 0 | |
| BaseBase Model=LLAMA3-8B-INSTRUCT2024.12 | 60.7 | 69.5 | 53.4 | 0 | |
| SFTBase Model=MISTRAL-7B-INSTRUCT2024.12 | 56.2 | 55.3 | 49.6 | 0.1 | |
| SFTBase Model=QWEN2-7B-INSTRUCT2024.12 | 55.7 | 55.1 | 49.7 | 0 | |
| BaseBase Model=QWEN2-7B-INSTRUCT2024.12 | 54.8 | 56.3 | 50.7 | 0 | |
| BaseBase Model=MISTRAL-7B-INSTRUCT2024.12 | 54.5 | 56.8 | 51 | 0.1 | |
| SFTBase Model=LLAMA2-7B-CHAT2024.12 | 51.5 | 40.4 | 43.9 | 2.2 | |
| BaseBase Model=LLAMA2-7B-CHAT2024.12 | 50.8 | 40.9 | 44.6 | 1.3 |